commit
083f1b589b
27 changed files with 2400 additions and 0 deletions
@ -0,0 +1,661 @@ |
|||||
|
GNU AFFERO GENERAL PUBLIC LICENSE |
||||
|
Version 3, 19 November 2007 |
||||
|
|
||||
|
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> |
||||
|
Everyone is permitted to copy and distribute verbatim copies |
||||
|
of this license document, but changing it is not allowed. |
||||
|
|
||||
|
Preamble |
||||
|
|
||||
|
The GNU Affero General Public License is a free, copyleft license for |
||||
|
software and other kinds of works, specifically designed to ensure |
||||
|
cooperation with the community in the case of network server software. |
||||
|
|
||||
|
The licenses for most software and other practical works are designed |
||||
|
to take away your freedom to share and change the works. By contrast, |
||||
|
our General Public Licenses are intended to guarantee your freedom to |
||||
|
share and change all versions of a program--to make sure it remains free |
||||
|
software for all its users. |
||||
|
|
||||
|
When we speak of free software, we are referring to freedom, not |
||||
|
price. Our General Public Licenses are designed to make sure that you |
||||
|
have the freedom to distribute copies of free software (and charge for |
||||
|
them if you wish), that you receive source code or can get it if you |
||||
|
want it, that you can change the software or use pieces of it in new |
||||
|
free programs, and that you know you can do these things. |
||||
|
|
||||
|
Developers that use our General Public Licenses protect your rights |
||||
|
with two steps: (1) assert copyright on the software, and (2) offer |
||||
|
you this License which gives you legal permission to copy, distribute |
||||
|
and/or modify the software. |
||||
|
|
||||
|
A secondary benefit of defending all users' freedom is that |
||||
|
improvements made in alternate versions of the program, if they |
||||
|
receive widespread use, become available for other developers to |
||||
|
incorporate. Many developers of free software are heartened and |
||||
|
encouraged by the resulting cooperation. However, in the case of |
||||
|
software used on network servers, this result may fail to come about. |
||||
|
The GNU General Public License permits making a modified version and |
||||
|
letting the public access it on a server without ever releasing its |
||||
|
source code to the public. |
||||
|
|
||||
|
The GNU Affero General Public License is designed specifically to |
||||
|
ensure that, in such cases, the modified source code becomes available |
||||
|
to the community. It requires the operator of a network server to |
||||
|
provide the source code of the modified version running there to the |
||||
|
users of that server. Therefore, public use of a modified version, on |
||||
|
a publicly accessible server, gives the public access to the source |
||||
|
code of the modified version. |
||||
|
|
||||
|
An older license, called the Affero General Public License and |
||||
|
published by Affero, was designed to accomplish similar goals. This is |
||||
|
a different license, not a version of the Affero GPL, but Affero has |
||||
|
released a new version of the Affero GPL which permits relicensing under |
||||
|
this license. |
||||
|
|
||||
|
The precise terms and conditions for copying, distribution and |
||||
|
modification follow. |
||||
|
|
||||
|
TERMS AND CONDITIONS |
||||
|
|
||||
|
0. Definitions. |
||||
|
|
||||
|
"This License" refers to version 3 of the GNU Affero General Public License. |
||||
|
|
||||
|
"Copyright" also means copyright-like laws that apply to other kinds of |
||||
|
works, such as semiconductor masks. |
||||
|
|
||||
|
"The Program" refers to any copyrightable work licensed under this |
||||
|
License. Each licensee is addressed as "you". "Licensees" and |
||||
|
"recipients" may be individuals or organizations. |
||||
|
|
||||
|
To "modify" a work means to copy from or adapt all or part of the work |
||||
|
in a fashion requiring copyright permission, other than the making of an |
||||
|
exact copy. The resulting work is called a "modified version" of the |
||||
|
earlier work or a work "based on" the earlier work. |
||||
|
|
||||
|
A "covered work" means either the unmodified Program or a work based |
||||
|
on the Program. |
||||
|
|
||||
|
To "propagate" a work means to do anything with it that, without |
||||
|
permission, would make you directly or secondarily liable for |
||||
|
infringement under applicable copyright law, except executing it on a |
||||
|
computer or modifying a private copy. Propagation includes copying, |
||||
|
distribution (with or without modification), making available to the |
||||
|
public, and in some countries other activities as well. |
||||
|
|
||||
|
To "convey" a work means any kind of propagation that enables other |
||||
|
parties to make or receive copies. Mere interaction with a user through |
||||
|
a computer network, with no transfer of a copy, is not conveying. |
||||
|
|
||||
|
An interactive user interface displays "Appropriate Legal Notices" |
||||
|
to the extent that it includes a convenient and prominently visible |
||||
|
feature that (1) displays an appropriate copyright notice, and (2) |
||||
|
tells the user that there is no warranty for the work (except to the |
||||
|
extent that warranties are provided), that licensees may convey the |
||||
|
work under this License, and how to view a copy of this License. If |
||||
|
the interface presents a list of user commands or options, such as a |
||||
|
menu, a prominent item in the list meets this criterion. |
||||
|
|
||||
|
1. Source Code. |
||||
|
|
||||
|
The "source code" for a work means the preferred form of the work |
||||
|
for making modifications to it. "Object code" means any non-source |
||||
|
form of a work. |
||||
|
|
||||
|
A "Standard Interface" means an interface that either is an official |
||||
|
standard defined by a recognized standards body, or, in the case of |
||||
|
interfaces specified for a particular programming language, one that |
||||
|
is widely used among developers working in that language. |
||||
|
|
||||
|
The "System Libraries" of an executable work include anything, other |
||||
|
than the work as a whole, that (a) is included in the normal form of |
||||
|
packaging a Major Component, but which is not part of that Major |
||||
|
Component, and (b) serves only to enable use of the work with that |
||||
|
Major Component, or to implement a Standard Interface for which an |
||||
|
implementation is available to the public in source code form. A |
||||
|
"Major Component", in this context, means a major essential component |
||||
|
(kernel, window system, and so on) of the specific operating system |
||||
|
(if any) on which the executable work runs, or a compiler used to |
||||
|
produce the work, or an object code interpreter used to run it. |
||||
|
|
||||
|
The "Corresponding Source" for a work in object code form means all |
||||
|
the source code needed to generate, install, and (for an executable |
||||
|
work) run the object code and to modify the work, including scripts to |
||||
|
control those activities. However, it does not include the work's |
||||
|
System Libraries, or general-purpose tools or generally available free |
||||
|
programs which are used unmodified in performing those activities but |
||||
|
which are not part of the work. For example, Corresponding Source |
||||
|
includes interface definition files associated with source files for |
||||
|
the work, and the source code for shared libraries and dynamically |
||||
|
linked subprograms that the work is specifically designed to require, |
||||
|
such as by intimate data communication or control flow between those |
||||
|
subprograms and other parts of the work. |
||||
|
|
||||
|
The Corresponding Source need not include anything that users |
||||
|
can regenerate automatically from other parts of the Corresponding |
||||
|
Source. |
||||
|
|
||||
|
The Corresponding Source for a work in source code form is that |
||||
|
same work. |
||||
|
|
||||
|
2. Basic Permissions. |
||||
|
|
||||
|
All rights granted under this License are granted for the term of |
||||
|
copyright on the Program, and are irrevocable provided the stated |
||||
|
conditions are met. This License explicitly affirms your unlimited |
||||
|
permission to run the unmodified Program. The output from running a |
||||
|
covered work is covered by this License only if the output, given its |
||||
|
content, constitutes a covered work. This License acknowledges your |
||||
|
rights of fair use or other equivalent, as provided by copyright law. |
||||
|
|
||||
|
You may make, run and propagate covered works that you do not |
||||
|
convey, without conditions so long as your license otherwise remains |
||||
|
in force. You may convey covered works to others for the sole purpose |
||||
|
of having them make modifications exclusively for you, or provide you |
||||
|
with facilities for running those works, provided that you comply with |
||||
|
the terms of this License in conveying all material for which you do |
||||
|
not control copyright. Those thus making or running the covered works |
||||
|
for you must do so exclusively on your behalf, under your direction |
||||
|
and control, on terms that prohibit them from making any copies of |
||||
|
your copyrighted material outside their relationship with you. |
||||
|
|
||||
|
Conveying under any other circumstances is permitted solely under |
||||
|
the conditions stated below. Sublicensing is not allowed; section 10 |
||||
|
makes it unnecessary. |
||||
|
|
||||
|
3. Protecting Users' Legal Rights From Anti-Circumvention Law. |
||||
|
|
||||
|
No covered work shall be deemed part of an effective technological |
||||
|
measure under any applicable law fulfilling obligations under article |
||||
|
11 of the WIPO copyright treaty adopted on 20 December 1996, or |
||||
|
similar laws prohibiting or restricting circumvention of such |
||||
|
measures. |
||||
|
|
||||
|
When you convey a covered work, you waive any legal power to forbid |
||||
|
circumvention of technological measures to the extent such circumvention |
||||
|
is effected by exercising rights under this License with respect to |
||||
|
the covered work, and you disclaim any intention to limit operation or |
||||
|
modification of the work as a means of enforcing, against the work's |
||||
|
users, your or third parties' legal rights to forbid circumvention of |
||||
|
technological measures. |
||||
|
|
||||
|
4. Conveying Verbatim Copies. |
||||
|
|
||||
|
You may convey verbatim copies of the Program's source code as you |
||||
|
receive it, in any medium, provided that you conspicuously and |
||||
|
appropriately publish on each copy an appropriate copyright notice; |
||||
|
keep intact all notices stating that this License and any |
||||
|
non-permissive terms added in accord with section 7 apply to the code; |
||||
|
keep intact all notices of the absence of any warranty; and give all |
||||
|
recipients a copy of this License along with the Program. |
||||
|
|
||||
|
You may charge any price or no price for each copy that you convey, |
||||
|
and you may offer support or warranty protection for a fee. |
||||
|
|
||||
|
5. Conveying Modified Source Versions. |
||||
|
|
||||
|
You may convey a work based on the Program, or the modifications to |
||||
|
produce it from the Program, in the form of source code under the |
||||
|
terms of section 4, provided that you also meet all of these conditions: |
||||
|
|
||||
|
a) The work must carry prominent notices stating that you modified |
||||
|
it, and giving a relevant date. |
||||
|
|
||||
|
b) The work must carry prominent notices stating that it is |
||||
|
released under this License and any conditions added under section |
||||
|
7. This requirement modifies the requirement in section 4 to |
||||
|
"keep intact all notices". |
||||
|
|
||||
|
c) You must license the entire work, as a whole, under this |
||||
|
License to anyone who comes into possession of a copy. This |
||||
|
License will therefore apply, along with any applicable section 7 |
||||
|
additional terms, to the whole of the work, and all its parts, |
||||
|
regardless of how they are packaged. This License gives no |
||||
|
permission to license the work in any other way, but it does not |
||||
|
invalidate such permission if you have separately received it. |
||||
|
|
||||
|
d) If the work has interactive user interfaces, each must display |
||||
|
Appropriate Legal Notices; however, if the Program has interactive |
||||
|
interfaces that do not display Appropriate Legal Notices, your |
||||
|
work need not make them do so. |
||||
|
|
||||
|
A compilation of a covered work with other separate and independent |
||||
|
works, which are not by their nature extensions of the covered work, |
||||
|
and which are not combined with it such as to form a larger program, |
||||
|
in or on a volume of a storage or distribution medium, is called an |
||||
|
"aggregate" if the compilation and its resulting copyright are not |
||||
|
used to limit the access or legal rights of the compilation's users |
||||
|
beyond what the individual works permit. Inclusion of a covered work |
||||
|
in an aggregate does not cause this License to apply to the other |
||||
|
parts of the aggregate. |
||||
|
|
||||
|
6. Conveying Non-Source Forms. |
||||
|
|
||||
|
You may convey a covered work in object code form under the terms |
||||
|
of sections 4 and 5, provided that you also convey the |
||||
|
machine-readable Corresponding Source under the terms of this License, |
||||
|
in one of these ways: |
||||
|
|
||||
|
a) Convey the object code in, or embodied in, a physical product |
||||
|
(including a physical distribution medium), accompanied by the |
||||
|
Corresponding Source fixed on a durable physical medium |
||||
|
customarily used for software interchange. |
||||
|
|
||||
|
b) Convey the object code in, or embodied in, a physical product |
||||
|
(including a physical distribution medium), accompanied by a |
||||
|
written offer, valid for at least three years and valid for as |
||||
|
long as you offer spare parts or customer support for that product |
||||
|
model, to give anyone who possesses the object code either (1) a |
||||
|
copy of the Corresponding Source for all the software in the |
||||
|
product that is covered by this License, on a durable physical |
||||
|
medium customarily used for software interchange, for a price no |
||||
|
more than your reasonable cost of physically performing this |
||||
|
conveying of source, or (2) access to copy the |
||||
|
Corresponding Source from a network server at no charge. |
||||
|
|
||||
|
c) Convey individual copies of the object code with a copy of the |
||||
|
written offer to provide the Corresponding Source. This |
||||
|
alternative is allowed only occasionally and noncommercially, and |
||||
|
only if you received the object code with such an offer, in accord |
||||
|
with subsection 6b. |
||||
|
|
||||
|
d) Convey the object code by offering access from a designated |
||||
|
place (gratis or for a charge), and offer equivalent access to the |
||||
|
Corresponding Source in the same way through the same place at no |
||||
|
further charge. You need not require recipients to copy the |
||||
|
Corresponding Source along with the object code. If the place to |
||||
|
copy the object code is a network server, the Corresponding Source |
||||
|
may be on a different server (operated by you or a third party) |
||||
|
that supports equivalent copying facilities, provided you maintain |
||||
|
clear directions next to the object code saying where to find the |
||||
|
Corresponding Source. Regardless of what server hosts the |
||||
|
Corresponding Source, you remain obligated to ensure that it is |
||||
|
available for as long as needed to satisfy these requirements. |
||||
|
|
||||
|
e) Convey the object code using peer-to-peer transmission, provided |
||||
|
you inform other peers where the object code and Corresponding |
||||
|
Source of the work are being offered to the general public at no |
||||
|
charge under subsection 6d. |
||||
|
|
||||
|
A separable portion of the object code, whose source code is excluded |
||||
|
from the Corresponding Source as a System Library, need not be |
||||
|
included in conveying the object code work. |
||||
|
|
||||
|
A "User Product" is either (1) a "consumer product", which means any |
||||
|
tangible personal property which is normally used for personal, family, |
||||
|
or household purposes, or (2) anything designed or sold for incorporation |
||||
|
into a dwelling. In determining whether a product is a consumer product, |
||||
|
doubtful cases shall be resolved in favor of coverage. For a particular |
||||
|
product received by a particular user, "normally used" refers to a |
||||
|
typical or common use of that class of product, regardless of the status |
||||
|
of the particular user or of the way in which the particular user |
||||
|
actually uses, or expects or is expected to use, the product. A product |
||||
|
is a consumer product regardless of whether the product has substantial |
||||
|
commercial, industrial or non-consumer uses, unless such uses represent |
||||
|
the only significant mode of use of the product. |
||||
|
|
||||
|
"Installation Information" for a User Product means any methods, |
||||
|
procedures, authorization keys, or other information required to install |
||||
|
and execute modified versions of a covered work in that User Product from |
||||
|
a modified version of its Corresponding Source. The information must |
||||
|
suffice to ensure that the continued functioning of the modified object |
||||
|
code is in no case prevented or interfered with solely because |
||||
|
modification has been made. |
||||
|
|
||||
|
If you convey an object code work under this section in, or with, or |
||||
|
specifically for use in, a User Product, and the conveying occurs as |
||||
|
part of a transaction in which the right of possession and use of the |
||||
|
User Product is transferred to the recipient in perpetuity or for a |
||||
|
fixed term (regardless of how the transaction is characterized), the |
||||
|
Corresponding Source conveyed under this section must be accompanied |
||||
|
by the Installation Information. But this requirement does not apply |
||||
|
if neither you nor any third party retains the ability to install |
||||
|
modified object code on the User Product (for example, the work has |
||||
|
been installed in ROM). |
||||
|
|
||||
|
The requirement to provide Installation Information does not include a |
||||
|
requirement to continue to provide support service, warranty, or updates |
||||
|
for a work that has been modified or installed by the recipient, or for |
||||
|
the User Product in which it has been modified or installed. Access to a |
||||
|
network may be denied when the modification itself materially and |
||||
|
adversely affects the operation of the network or violates the rules and |
||||
|
protocols for communication across the network. |
||||
|
|
||||
|
Corresponding Source conveyed, and Installation Information provided, |
||||
|
in accord with this section must be in a format that is publicly |
||||
|
documented (and with an implementation available to the public in |
||||
|
source code form), and must require no special password or key for |
||||
|
unpacking, reading or copying. |
||||
|
|
||||
|
7. Additional Terms. |
||||
|
|
||||
|
"Additional permissions" are terms that supplement the terms of this |
||||
|
License by making exceptions from one or more of its conditions. |
||||
|
Additional permissions that are applicable to the entire Program shall |
||||
|
be treated as though they were included in this License, to the extent |
||||
|
that they are valid under applicable law. If additional permissions |
||||
|
apply only to part of the Program, that part may be used separately |
||||
|
under those permissions, but the entire Program remains governed by |
||||
|
this License without regard to the additional permissions. |
||||
|
|
||||
|
When you convey a copy of a covered work, you may at your option |
||||
|
remove any additional permissions from that copy, or from any part of |
||||
|
it. (Additional permissions may be written to require their own |
||||
|
removal in certain cases when you modify the work.) You may place |
||||
|
additional permissions on material, added by you to a covered work, |
||||
|
for which you have or can give appropriate copyright permission. |
||||
|
|
||||
|
Notwithstanding any other provision of this License, for material you |
||||
|
add to a covered work, you may (if authorized by the copyright holders of |
||||
|
that material) supplement the terms of this License with terms: |
||||
|
|
||||
|
a) Disclaiming warranty or limiting liability differently from the |
||||
|
terms of sections 15 and 16 of this License; or |
||||
|
|
||||
|
b) Requiring preservation of specified reasonable legal notices or |
||||
|
author attributions in that material or in the Appropriate Legal |
||||
|
Notices displayed by works containing it; or |
||||
|
|
||||
|
c) Prohibiting misrepresentation of the origin of that material, or |
||||
|
requiring that modified versions of such material be marked in |
||||
|
reasonable ways as different from the original version; or |
||||
|
|
||||
|
d) Limiting the use for publicity purposes of names of licensors or |
||||
|
authors of the material; or |
||||
|
|
||||
|
e) Declining to grant rights under trademark law for use of some |
||||
|
trade names, trademarks, or service marks; or |
||||
|
|
||||
|
f) Requiring indemnification of licensors and authors of that |
||||
|
material by anyone who conveys the material (or modified versions of |
||||
|
it) with contractual assumptions of liability to the recipient, for |
||||
|
any liability that these contractual assumptions directly impose on |
||||
|
those licensors and authors. |
||||
|
|
||||
|
All other non-permissive additional terms are considered "further |
||||
|
restrictions" within the meaning of section 10. If the Program as you |
||||
|
received it, or any part of it, contains a notice stating that it is |
||||
|
governed by this License along with a term that is a further |
||||
|
restriction, you may remove that term. If a license document contains |
||||
|
a further restriction but permits relicensing or conveying under this |
||||
|
License, you may add to a covered work material governed by the terms |
||||
|
of that license document, provided that the further restriction does |
||||
|
not survive such relicensing or conveying. |
||||
|
|
||||
|
If you add terms to a covered work in accord with this section, you |
||||
|
must place, in the relevant source files, a statement of the |
||||
|
additional terms that apply to those files, or a notice indicating |
||||
|
where to find the applicable terms. |
||||
|
|
||||
|
Additional terms, permissive or non-permissive, may be stated in the |
||||
|
form of a separately written license, or stated as exceptions; |
||||
|
the above requirements apply either way. |
||||
|
|
||||
|
8. Termination. |
||||
|
|
||||
|
You may not propagate or modify a covered work except as expressly |
||||
|
provided under this License. Any attempt otherwise to propagate or |
||||
|
modify it is void, and will automatically terminate your rights under |
||||
|
this License (including any patent licenses granted under the third |
||||
|
paragraph of section 11). |
||||
|
|
||||
|
However, if you cease all violation of this License, then your |
||||
|
license from a particular copyright holder is reinstated (a) |
||||
|
provisionally, unless and until the copyright holder explicitly and |
||||
|
finally terminates your license, and (b) permanently, if the copyright |
||||
|
holder fails to notify you of the violation by some reasonable means |
||||
|
prior to 60 days after the cessation. |
||||
|
|
||||
|
Moreover, your license from a particular copyright holder is |
||||
|
reinstated permanently if the copyright holder notifies you of the |
||||
|
violation by some reasonable means, this is the first time you have |
||||
|
received notice of violation of this License (for any work) from that |
||||
|
copyright holder, and you cure the violation prior to 30 days after |
||||
|
your receipt of the notice. |
||||
|
|
||||
|
Termination of your rights under this section does not terminate the |
||||
|
licenses of parties who have received copies or rights from you under |
||||
|
this License. If your rights have been terminated and not permanently |
||||
|
reinstated, you do not qualify to receive new licenses for the same |
||||
|
material under section 10. |
||||
|
|
||||
|
9. Acceptance Not Required for Having Copies. |
||||
|
|
||||
|
You are not required to accept this License in order to receive or |
||||
|
run a copy of the Program. Ancillary propagation of a covered work |
||||
|
occurring solely as a consequence of using peer-to-peer transmission |
||||
|
to receive a copy likewise does not require acceptance. However, |
||||
|
nothing other than this License grants you permission to propagate or |
||||
|
modify any covered work. These actions infringe copyright if you do |
||||
|
not accept this License. Therefore, by modifying or propagating a |
||||
|
covered work, you indicate your acceptance of this License to do so. |
||||
|
|
||||
|
10. Automatic Licensing of Downstream Recipients. |
||||
|
|
||||
|
Each time you convey a covered work, the recipient automatically |
||||
|
receives a license from the original licensors, to run, modify and |
||||
|
propagate that work, subject to this License. You are not responsible |
||||
|
for enforcing compliance by third parties with this License. |
||||
|
|
||||
|
An "entity transaction" is a transaction transferring control of an |
||||
|
organization, or substantially all assets of one, or subdividing an |
||||
|
organization, or merging organizations. If propagation of a covered |
||||
|
work results from an entity transaction, each party to that |
||||
|
transaction who receives a copy of the work also receives whatever |
||||
|
licenses to the work the party's predecessor in interest had or could |
||||
|
give under the previous paragraph, plus a right to possession of the |
||||
|
Corresponding Source of the work from the predecessor in interest, if |
||||
|
the predecessor has it or can get it with reasonable efforts. |
||||
|
|
||||
|
You may not impose any further restrictions on the exercise of the |
||||
|
rights granted or affirmed under this License. For example, you may |
||||
|
not impose a license fee, royalty, or other charge for exercise of |
||||
|
rights granted under this License, and you may not initiate litigation |
||||
|
(including a cross-claim or counterclaim in a lawsuit) alleging that |
||||
|
any patent claim is infringed by making, using, selling, offering for |
||||
|
sale, or importing the Program or any portion of it. |
||||
|
|
||||
|
11. Patents. |
||||
|
|
||||
|
A "contributor" is a copyright holder who authorizes use under this |
||||
|
License of the Program or a work on which the Program is based. The |
||||
|
work thus licensed is called the contributor's "contributor version". |
||||
|
|
||||
|
A contributor's "essential patent claims" are all patent claims |
||||
|
owned or controlled by the contributor, whether already acquired or |
||||
|
hereafter acquired, that would be infringed by some manner, permitted |
||||
|
by this License, of making, using, or selling its contributor version, |
||||
|
but do not include claims that would be infringed only as a |
||||
|
consequence of further modification of the contributor version. For |
||||
|
purposes of this definition, "control" includes the right to grant |
||||
|
patent sublicenses in a manner consistent with the requirements of |
||||
|
this License. |
||||
|
|
||||
|
Each contributor grants you a non-exclusive, worldwide, royalty-free |
||||
|
patent license under the contributor's essential patent claims, to |
||||
|
make, use, sell, offer for sale, import and otherwise run, modify and |
||||
|
propagate the contents of its contributor version. |
||||
|
|
||||
|
In the following three paragraphs, a "patent license" is any express |
||||
|
agreement or commitment, however denominated, not to enforce a patent |
||||
|
(such as an express permission to practice a patent or covenant not to |
||||
|
sue for patent infringement). To "grant" such a patent license to a |
||||
|
party means to make such an agreement or commitment not to enforce a |
||||
|
patent against the party. |
||||
|
|
||||
|
If you convey a covered work, knowingly relying on a patent license, |
||||
|
and the Corresponding Source of the work is not available for anyone |
||||
|
to copy, free of charge and under the terms of this License, through a |
||||
|
publicly available network server or other readily accessible means, |
||||
|
then you must either (1) cause the Corresponding Source to be so |
||||
|
available, or (2) arrange to deprive yourself of the benefit of the |
||||
|
patent license for this particular work, or (3) arrange, in a manner |
||||
|
consistent with the requirements of this License, to extend the patent |
||||
|
license to downstream recipients. "Knowingly relying" means you have |
||||
|
actual knowledge that, but for the patent license, your conveying the |
||||
|
covered work in a country, or your recipient's use of the covered work |
||||
|
in a country, would infringe one or more identifiable patents in that |
||||
|
country that you have reason to believe are valid. |
||||
|
|
||||
|
If, pursuant to or in connection with a single transaction or |
||||
|
arrangement, you convey, or propagate by procuring conveyance of, a |
||||
|
covered work, and grant a patent license to some of the parties |
||||
|
receiving the covered work authorizing them to use, propagate, modify |
||||
|
or convey a specific copy of the covered work, then the patent license |
||||
|
you grant is automatically extended to all recipients of the covered |
||||
|
work and works based on it. |
||||
|
|
||||
|
A patent license is "discriminatory" if it does not include within |
||||
|
the scope of its coverage, prohibits the exercise of, or is |
||||
|
conditioned on the non-exercise of one or more of the rights that are |
||||
|
specifically granted under this License. You may not convey a covered |
||||
|
work if you are a party to an arrangement with a third party that is |
||||
|
in the business of distributing software, under which you make payment |
||||
|
to the third party based on the extent of your activity of conveying |
||||
|
the work, and under which the third party grants, to any of the |
||||
|
parties who would receive the covered work from you, a discriminatory |
||||
|
patent license (a) in connection with copies of the covered work |
||||
|
conveyed by you (or copies made from those copies), or (b) primarily |
||||
|
for and in connection with specific products or compilations that |
||||
|
contain the covered work, unless you entered into that arrangement, |
||||
|
or that patent license was granted, prior to 28 March 2007. |
||||
|
|
||||
|
Nothing in this License shall be construed as excluding or limiting |
||||
|
any implied license or other defenses to infringement that may |
||||
|
otherwise be available to you under applicable patent law. |
||||
|
|
||||
|
12. No Surrender of Others' Freedom. |
||||
|
|
||||
|
If conditions are imposed on you (whether by court order, agreement or |
||||
|
otherwise) that contradict the conditions of this License, they do not |
||||
|
excuse you from the conditions of this License. If you cannot convey a |
||||
|
covered work so as to satisfy simultaneously your obligations under this |
||||
|
License and any other pertinent obligations, then as a consequence you may |
||||
|
not convey it at all. For example, if you agree to terms that obligate you |
||||
|
to collect a royalty for further conveying from those to whom you convey |
||||
|
the Program, the only way you could satisfy both those terms and this |
||||
|
License would be to refrain entirely from conveying the Program. |
||||
|
|
||||
|
13. Remote Network Interaction; Use with the GNU General Public License. |
||||
|
|
||||
|
Notwithstanding any other provision of this License, if you modify the |
||||
|
Program, your modified version must prominently offer all users |
||||
|
interacting with it remotely through a computer network (if your version |
||||
|
supports such interaction) an opportunity to receive the Corresponding |
||||
|
Source of your version by providing access to the Corresponding Source |
||||
|
from a network server at no charge, through some standard or customary |
||||
|
means of facilitating copying of software. This Corresponding Source |
||||
|
shall include the Corresponding Source for any work covered by version 3 |
||||
|
of the GNU General Public License that is incorporated pursuant to the |
||||
|
following paragraph. |
||||
|
|
||||
|
Notwithstanding any other provision of this License, you have |
||||
|
permission to link or combine any covered work with a work licensed |
||||
|
under version 3 of the GNU General Public License into a single |
||||
|
combined work, and to convey the resulting work. The terms of this |
||||
|
License will continue to apply to the part which is the covered work, |
||||
|
but the work with which it is combined will remain governed by version |
||||
|
3 of the GNU General Public License. |
||||
|
|
||||
|
14. Revised Versions of this License. |
||||
|
|
||||
|
The Free Software Foundation may publish revised and/or new versions of |
||||
|
the GNU Affero General Public License from time to time. Such new versions |
||||
|
will be similar in spirit to the present version, but may differ in detail to |
||||
|
address new problems or concerns. |
||||
|
|
||||
|
Each version is given a distinguishing version number. If the |
||||
|
Program specifies that a certain numbered version of the GNU Affero General |
||||
|
Public License "or any later version" applies to it, you have the |
||||
|
option of following the terms and conditions either of that numbered |
||||
|
version or of any later version published by the Free Software |
||||
|
Foundation. If the Program does not specify a version number of the |
||||
|
GNU Affero General Public License, you may choose any version ever published |
||||
|
by the Free Software Foundation. |
||||
|
|
||||
|
If the Program specifies that a proxy can decide which future |
||||
|
versions of the GNU Affero General Public License can be used, that proxy's |
||||
|
public statement of acceptance of a version permanently authorizes you |
||||
|
to choose that version for the Program. |
||||
|
|
||||
|
Later license versions may give you additional or different |
||||
|
permissions. However, no additional obligations are imposed on any |
||||
|
author or copyright holder as a result of your choosing to follow a |
||||
|
later version. |
||||
|
|
||||
|
15. Disclaimer of Warranty. |
||||
|
|
||||
|
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY |
||||
|
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT |
||||
|
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY |
||||
|
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, |
||||
|
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR |
||||
|
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM |
||||
|
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF |
||||
|
ALL NECESSARY SERVICING, REPAIR OR CORRECTION. |
||||
|
|
||||
|
16. Limitation of Liability. |
||||
|
|
||||
|
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING |
||||
|
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS |
||||
|
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY |
||||
|
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE |
||||
|
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF |
||||
|
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD |
||||
|
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), |
||||
|
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF |
||||
|
SUCH DAMAGES. |
||||
|
|
||||
|
17. Interpretation of Sections 15 and 16. |
||||
|
|
||||
|
If the disclaimer of warranty and limitation of liability provided |
||||
|
above cannot be given local legal effect according to their terms, |
||||
|
reviewing courts shall apply local law that most closely approximates |
||||
|
an absolute waiver of all civil liability in connection with the |
||||
|
Program, unless a warranty or assumption of liability accompanies a |
||||
|
copy of the Program in return for a fee. |
||||
|
|
||||
|
END OF TERMS AND CONDITIONS |
||||
|
|
||||
|
How to Apply These Terms to Your New Programs |
||||
|
|
||||
|
If you develop a new program, and you want it to be of the greatest |
||||
|
possible use to the public, the best way to achieve this is to make it |
||||
|
free software which everyone can redistribute and change under these terms. |
||||
|
|
||||
|
To do so, attach the following notices to the program. It is safest |
||||
|
to attach them to the start of each source file to most effectively |
||||
|
state the exclusion of warranty; and each file should have at least |
||||
|
the "copyright" line and a pointer to where the full notice is found. |
||||
|
|
||||
|
<one line to give the program's name and a brief idea of what it does.> |
||||
|
Copyright (C) <year> <name of author> |
||||
|
|
||||
|
This program is free software: you can redistribute it and/or modify |
||||
|
it under the terms of the GNU Affero General Public License as published by |
||||
|
the Free Software Foundation, either version 3 of the License, or |
||||
|
(at your option) any later version. |
||||
|
|
||||
|
This program is distributed in the hope that it will be useful, |
||||
|
but WITHOUT ANY WARRANTY; without even the implied warranty of |
||||
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
||||
|
GNU Affero General Public License for more details. |
||||
|
|
||||
|
You should have received a copy of the GNU Affero General Public License |
||||
|
along with this program. If not, see <http://www.gnu.org/licenses/>. |
||||
|
|
||||
|
Also add information on how to contact you by electronic and paper mail. |
||||
|
|
||||
|
If your software can interact with users remotely through a computer |
||||
|
network, you should also make sure that it provides a way for users to |
||||
|
get its source. For example, if your program is a web application, its |
||||
|
interface could display a "Source" link that leads users to an archive |
||||
|
of the code. There are many ways you could offer source, and different |
||||
|
solutions will be better for different programs; see section 13 for the |
||||
|
specific requirements. |
||||
|
|
||||
|
You should also get your employer (if you work as a programmer) or school, |
||||
|
if any, to sign a "copyright disclaimer" for the program, if necessary. |
||||
|
For more information on this, and how to apply and follow the GNU AGPL, see |
||||
|
<http://www.gnu.org/licenses/>. |
||||
@ -0,0 +1,11 @@ |
|||||
|
A pluggable search engine designed to find alternative copies of textbooks. |
||||
|
Allows you to plug in a python module that provides textbook data for your |
||||
|
institution, then indexes that data in elasticsearch and lets people search for |
||||
|
books. It will try to find alternatives through various means. At the moment it |
||||
|
only searches the Internet Archive, and Open Library, however the plan is to |
||||
|
refine it to find better matches. |
||||
|
|
||||
|
Code is available under the GNU Affero General Public License |
||||
|
This means you must make any changes you make available if you run it on your |
||||
|
own server. See [here](https://www.gnu.org/licenses/why-affero-gpl.html) for |
||||
|
more info. |
||||
@ -0,0 +1,34 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
from urllib import quote |
||||
|
from json import loads, dumps |
||||
|
|
||||
|
import requests as req |
||||
|
|
||||
|
searchUrl = "https://archive.org/advancedsearch.php?q={0}&fl%5B%5D=avg_rating&fl%5B%5D=description&fl%5B%5D=identifier&fl%5B%5D=type&sort%5B%5D=&sort%5B%5D=&sort%5B%5D=&rows=50&page=1&output=json&callback=callback&save=yes#raw" |
||||
|
|
||||
|
def searchIA(title, author): |
||||
|
""" |
||||
|
Do a search on The Internet Archive for a book |
||||
|
""" |
||||
|
print "running a search" |
||||
|
requrl = searchUrl.format(quote(title + " " + author)) |
||||
|
try: |
||||
|
results = loads(req.get(requrl).text[9:][0:-1]) |
||||
|
except ValueError: |
||||
|
return [] |
||||
|
|
||||
|
rownum = results["responseHeader"]["params"]["rows"] |
||||
|
if rownum < 1: |
||||
|
print "Couldn't find results for %s %s" % (title, author) |
||||
|
return [] |
||||
|
docs = results["response"]["docs"] |
||||
|
urls = [] |
||||
|
for result in results["response"]["docs"][0:3]: |
||||
|
urls.append("https://archive.org/details/%s" % result["identifier"]) |
||||
|
return urls |
||||
|
|
||||
|
|
||||
|
# Example, search for David Hume's Enquiry Concerning Human Understanding |
||||
|
#for url in searchIA("Hume", "Enquiry Concerning Human Understanding"): |
||||
|
#print url |
||||
@ -0,0 +1 @@ |
|||||
|
{"course":{"properties":{"textbooks":{"properties":{"author":{"type":"string","index":"analyzed"},"price":{"type":"string","index":"analyzed"},"title":{"type":"string","index":"analyzed"}}},"sections":{"properties":{"time":{"type":"string","index":"analyzed"},"title":{"type":"string","index":"analyzed"},"loc":{"type":"string","index":"analyzed"},"prof":{"type":"string","index":"analyzed"},"sem":{"type":"string","index":"analyzed"},"day":{"type":"string","index":"analyzed"}}}}}} |
||||
@ -0,0 +1,19 @@ |
|||||
|
#! /usr/bin/racket |
||||
|
#lang racket |
||||
|
(require "schemadsl.rkt") |
||||
|
|
||||
|
(displayln |
||||
|
(make-mapping |
||||
|
"course" |
||||
|
`(,(estruct "sections" |
||||
|
`(,(str "title") |
||||
|
,(str "time") |
||||
|
,(str "loc") |
||||
|
,(str "prof") |
||||
|
,(str "sem") |
||||
|
,(str "day"))) |
||||
|
|
||||
|
,(estruct "textbooks" |
||||
|
`(,(str "title") |
||||
|
,(str "author") |
||||
|
,(str "price")))))) |
||||
@ -0,0 +1,62 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
from sys import argv |
||||
|
from hashlib import sha1 |
||||
|
|
||||
|
def truncate(docid): |
||||
|
""" |
||||
|
Truncate a document id to 12 digits |
||||
|
The document ID should be based on a |
||||
|
hash of unique identifiers |
||||
|
""" |
||||
|
return int(str(docid)[0:12]) |
||||
|
|
||||
|
def createResource(textbookInfo, course, dept, coursecode, docid): |
||||
|
""" |
||||
|
Create a document associated with a course |
||||
|
This document contains any/all resources associated |
||||
|
with that course |
||||
|
|
||||
|
example, |
||||
|
{ |
||||
|
'books': [], |
||||
|
'dept': 'COLLAB', |
||||
|
'code': '2C03', |
||||
|
'sections': [ |
||||
|
{ |
||||
|
'prof': 'Lisa Pender', |
||||
|
'sem': '2015/09/08 - 2015/12/08', |
||||
|
'day': 'Mo' |
||||
|
}, |
||||
|
{ |
||||
|
'prof': 'Staff', |
||||
|
'sem': '2015/09/08 - 2015/12/08', |
||||
|
'day': 'Th' |
||||
|
} |
||||
|
], |
||||
|
'title': 'COLLAB 2C03 - Sociology I' |
||||
|
} |
||||
|
""" |
||||
|
textbooks = textbookInfo(dept.strip(), coursecode.strip()) |
||||
|
|
||||
|
# We truncate the id so we can have nicer looking URLs |
||||
|
# Since the id will be used to point to the resource page for that course |
||||
|
_id = str(truncate(docid)) |
||||
|
|
||||
|
fields = { |
||||
|
"_id" : _id, |
||||
|
"textbooks" : textbooks, |
||||
|
"coursetitle" : "%s %s" % (dept.strip(), coursecode.strip()), |
||||
|
"courseinfo" : course |
||||
|
#"Syllabus" : "blah" |
||||
|
} |
||||
|
try: |
||||
|
revisions = list(localdb.revisions(_id)) |
||||
|
if not revisions: |
||||
|
return localdb.save(fields) |
||||
|
else: |
||||
|
rev = dict(revisions[0])["_rev"] |
||||
|
fields["_rev"] = rev |
||||
|
return localdb.save(fields) |
||||
|
except ResourceConflict: |
||||
|
print "Resource for %s already exists, not creating a new one" % (docid) |
||||
@ -0,0 +1,14 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
# predictive data |
||||
|
# switch to elasticsearch's prediction |
||||
|
|
||||
|
|
||||
|
|
||||
|
import database |
||||
|
import predictions |
||||
|
|
||||
|
class GOASearch(object): |
||||
|
def __init__(self): |
||||
|
return self |
||||
|
|
||||
@ -0,0 +1,349 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
from sys import argv |
||||
|
from itertools import chain, islice, izip as zip |
||||
|
from re import search, sub |
||||
|
from functools import total_ordering |
||||
|
|
||||
|
from sylla import textbookInfo |
||||
|
from collections import MutableMapping |
||||
|
|
||||
|
import datetime as dt |
||||
|
import lxml.html as lxh |
||||
|
import requests |
||||
|
import sys |
||||
|
import copy |
||||
|
|
||||
|
fall = "2159" |
||||
|
spring_summer = "2165" |
||||
|
winter = "2161" |
||||
|
|
||||
|
# threading stuff |
||||
|
import Queue as q |
||||
|
import threading as thd |
||||
|
|
||||
|
baseurl = "https://applicants.mcmaster.ca/psp/prepprd/EMPLOYEE/PSFT_LS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL" |
||||
|
|
||||
|
searchurl = "https://csprd.mcmaster.ca/psc/prcsprd/EMPLOYEE/PSFT_LS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL" |
||||
|
|
||||
|
custom_headers = { |
||||
|
"User-Agent" : "Mozilla/5.0 (X11; Linux x86_64; rv:41.0) Gecko/20100101 Firefox/41.0", |
||||
|
"Content-Type" : "application/x-www-form-urlencoded; charset=UTF-8", |
||||
|
} |
||||
|
|
||||
|
courseCodes1 = "ICAJAX=1&ICNAVTYPEDROPDOWN=1&ICType=Panel&ICElementNum=0&ICStateNum={0}&ICAction=CLASS_SRCH_WRK2_SSR_PB_SUBJ_SRCH%240&ICXPos=0&ICYPos=0&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICResubmit=0&ICSID=5tq9x%2Fjt42mf62Sh5z%2BrjxT0gT15kiIyQ2cecCSmRB4%3D&ICActionPrompt=false&ICFind=&ICAddCount=&ICAPPCLSDATA=&CLASS_SRCH_WRK2_STRM$45$={1}" |
||||
|
|
||||
|
courseCodes2 = "ICAJAX=1&ICNAVTYPEDROPDOWN=1&ICType=Panel&ICElementNum=0&ICStateNum={0}&ICAction=SSR_CLSRCH_WRK2_SSR_ALPHANUM_{1}&ICXPos=0&ICYPos=0&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICResubmit=0&ICSID=vIUgl6ZXw045S07EPbQw4RDzv7NmKCDdJFdT4CTRQNM%3D&ICActionPrompt=false&ICFind=&ICAddCount=&ICAPPCLSDATA=&CLASS_SRCH_WRK2_STRM$45$={2}" |
||||
|
|
||||
|
payload2 = "ICAJAX=1&ICNAVTYPEDROPDOWN=1&ICType=Panel&ICElementNum=0&ICStateNum={0}&ICAction=%23ICSave&ICXPos=0&ICYPos=0&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICResubmit=0&ICSID=aWx3w6lJ6d2wZui6hwRVSEnzsPgCA3afYJEFBLLkxe4%3D&ICActionPrompt=false&ICFind=&ICAddCount=&ICAPPCLSDATA=&CLASS_SRCH_WRK2_STRM$45$={1}" |
||||
|
|
||||
|
payload = "ICAJAX=1&ICNAVTYPEDROPDOWN=1&ICType=Panel&ICElementNum=0&ICStateNum={0}&ICAction=CLASS_SRCH_WRK2_SSR_PB_CLASS_SRCH&ICXPos=0&ICYPos=0&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICResubmit=0&ICSID=aWx3w6lJ6d2wZui6hwRVSEnzsPgCA3afYJEFBLLkxe4%3D&ICActionPrompt=false&ICFind=&ICAddCount=&ICAPPCLSDATA=&SSR_CLSRCH_WRK_SUBJECT$75$$0={1}&CLASS_SRCH_WRK2_STRM$45$={2}" |
||||
|
|
||||
|
|
||||
|
year = dt.date.today().year |
||||
|
month = dt.date.today().month |
||||
|
|
||||
|
days = { |
||||
|
"Mo" : 0, |
||||
|
"Tu" : 1, |
||||
|
"We" : 2, |
||||
|
"Th" : 3, |
||||
|
"Fr" : 4, |
||||
|
"Sa" : 5, |
||||
|
"Su" : 6 |
||||
|
} |
||||
|
|
||||
|
day_descs = { |
||||
|
"Mo" : "Monday Mon Mo", |
||||
|
"Tu" : "Tuesday Tues Tu Tue", |
||||
|
"We" : "Wednesday Wed We", |
||||
|
"Th" : "Thursday Th Thurs", |
||||
|
"Fr" : "Friday Fr Fri", |
||||
|
"Sa" : "Saturday Sat Sa", |
||||
|
"Su" : "Sunday Su Sun", |
||||
|
"T" : "TBA" |
||||
|
} |
||||
|
|
||||
|
def timeparse(time): |
||||
|
""" |
||||
|
Parse the time into numbers |
||||
|
""" |
||||
|
if len(time) == 7: |
||||
|
hour = int(time[0:2]) |
||||
|
minutes = int(time[3:5]) |
||||
|
half = time[5:7] |
||||
|
else: |
||||
|
hour = int(time[0]) |
||||
|
minutes = int(time[2:4]) |
||||
|
half = time[4:6] |
||||
|
if half == "PM": |
||||
|
if hour < 12: |
||||
|
hour = hour + 12 |
||||
|
|
||||
|
return (str(hour), str(minutes), half) |
||||
|
|
||||
|
class Class(object): |
||||
|
def __init__(self, dept, title, sections): |
||||
|
self.title = title.encode("UTF-8") |
||||
|
self.sections = sections |
||||
|
self.dept = dept |
||||
|
|
||||
|
def __repr__(self): |
||||
|
return repr((self.title, self.sections)) |
||||
|
|
||||
|
def __iter__(self): |
||||
|
return iter((self.title, sec) for sec in self.sections) |
||||
|
|
||||
|
def hasCode(self): |
||||
|
splitted = self.title.strip().split(" ") |
||||
|
return ((len(splitted) >= 2) and |
||||
|
(splitted[0].upper() == splitted[0]) and |
||||
|
(splitted[1].upper() == splitted[1])) |
||||
|
|
||||
|
@property |
||||
|
def code(self): |
||||
|
if self.hasCode(): |
||||
|
return self.title.strip().split(" ")[1].strip() |
||||
|
return False |
||||
|
|
||||
|
@property |
||||
|
def books(self): |
||||
|
if self.dept and self.code: |
||||
|
return textbookInfo(self.dept, self.code, withPrices=True) |
||||
|
return False |
||||
|
|
||||
|
@total_ordering |
||||
|
class Section(dict): |
||||
|
def __init__(self, time, loc, prof, sem): |
||||
|
self.time = time.encode("UTF-8") |
||||
|
self.loc = loc.encode("UTF-8") |
||||
|
self.prof = prof.encode("UTF-8") |
||||
|
self.sem = sem.encode("UTF-8") |
||||
|
self._date = False |
||||
|
self._day = False |
||||
|
|
||||
|
@property |
||||
|
def date(self): |
||||
|
if self.time != "TBA": |
||||
|
day, start, _, end = self.time.split() |
||||
|
|
||||
|
if self._day: |
||||
|
assert len(self._day) == 2 |
||||
|
day = self._day |
||||
|
else: |
||||
|
day = [day[n:n+2] for n in xrange(0, len(day)-1, 2)] |
||||
|
|
||||
|
self._date = (day, timeparse(start), timeparse(end)) |
||||
|
|
||||
|
return self._date |
||||
|
|
||||
|
return self.time |
||||
|
|
||||
|
@property |
||||
|
def day(self): |
||||
|
return self.date[0] |
||||
|
|
||||
|
@property |
||||
|
def start(self): |
||||
|
return self.date[1][0] + self.date[1][1] |
||||
|
|
||||
|
def __repr__(self): |
||||
|
return (""" |
||||
|
Time = %s, Location = %s, Instructor = %s, Semester Running = %s |
||||
|
""" % (self.date, self.loc, self.prof, self.sem)) |
||||
|
def __gt__(self, x): |
||||
|
if isinstance(self.day, list): |
||||
|
raise NotImplementedError |
||||
|
|
||||
|
if (self.date == "TBA" or |
||||
|
x.date == "TBA"): |
||||
|
return False |
||||
|
|
||||
|
return ((days[self.day] > days[x.day]) or |
||||
|
((self.day == x.day) and |
||||
|
(self.start > x.start))) |
||||
|
|
||||
|
def __eq__(self, x): |
||||
|
return (x.date == self.date and |
||||
|
x.prof == self.prof and |
||||
|
x.loc == self.loc and |
||||
|
x.sem == self.sem) |
||||
|
|
||||
|
|
||||
|
def getStateNum(html): |
||||
|
""" |
||||
|
Get the state num from Mosaic |
||||
|
This is unique to each requester |
||||
|
""" |
||||
|
parsed = lxh.fromstring(html) |
||||
|
return parsed.xpath(".//input[@name=\"ICStateNum\"]")[0].value |
||||
|
|
||||
|
def parseSection(section): |
||||
|
cols = section.xpath(".//td") |
||||
|
assert len(cols) == 4 |
||||
|
time, loc, prof, sem = [col.text_content().encode("UTF-8").strip() for col in cols] |
||||
|
|
||||
|
classinfo = Section(time, loc, prof, sem) |
||||
|
return classinfo |
||||
|
|
||||
|
def getSectionInfo(table): |
||||
|
trs = table.xpath(".//tr") |
||||
|
for tr in trs: |
||||
|
if tr.xpath("@id") and search(r"SSR_CLSRCH", tr.xpath("@id")[0]): |
||||
|
yield parseSection(tr) |
||||
|
|
||||
|
def parseColumns(subject, html): |
||||
|
parsed = lxh.fromstring(html) |
||||
|
|
||||
|
classInfo = (list(getSectionInfo(table)) for table in |
||||
|
islice((table for table in parsed.xpath(".//table") |
||||
|
if table.xpath("@id") and |
||||
|
search(r"ICField[0-9]+\$scroll", table.xpath("@id")[0])), 1, sys.maxint)) |
||||
|
|
||||
|
classNames = ((subject, span.text_content().strip()) for span in parsed.xpath(".//span") |
||||
|
if span.xpath("@id") and |
||||
|
search(r"DERIVED_CLSRCH_DESCR", span.xpath("@id")[0])) |
||||
|
|
||||
|
return zip(classNames, classInfo) |
||||
|
|
||||
|
def getCodes(html): |
||||
|
parsed = lxh.fromstring(html) |
||||
|
|
||||
|
return (code.text_content().encode("UTF-8") for code in |
||||
|
parsed.xpath("//span") |
||||
|
if code.xpath("@id") and |
||||
|
search(r"SSR_CLSRCH_SUBJ_SUBJECT\$[0-9]+", code.xpath("@id")[0])) |
||||
|
|
||||
|
class MosReq(object): |
||||
|
def __init__(self, semester): |
||||
|
self.semester = semester |
||||
|
s = requests.Session() |
||||
|
resp = s.get(baseurl, allow_redirects=True, headers=custom_headers).content |
||||
|
|
||||
|
# Let the server set some cookies before doing the searching |
||||
|
cookies = {} |
||||
|
for key, val in s.cookies.iteritems(): |
||||
|
cookies[key] = val |
||||
|
self.cookies = cookies |
||||
|
self.statenum = False |
||||
|
self.codes_ = [] |
||||
|
|
||||
|
def getlist(self, subject): |
||||
|
sys.stderr.write("Getting " + subject + "\n") |
||||
|
first_req = requests.get(searchurl, cookies=self.cookies).content |
||||
|
# for some reason Mosaic wants us to request it twice, ?????????????????? |
||||
|
self.statenum = getStateNum(first_req) |
||||
|
first_req = requests.post(searchurl, |
||||
|
data=payload.format(self.statenum, subject, self.semester), |
||||
|
cookies=self.cookies, |
||||
|
allow_redirects=False, |
||||
|
headers=custom_headers).content |
||||
|
# we make a first request to get the ICStateNum in case it thinks there are too many results |
||||
|
try: |
||||
|
self.statenum = getStateNum(first_req) |
||||
|
except IndexError: |
||||
|
pass |
||||
|
if "Your search will return over" in first_req: |
||||
|
|
||||
|
return requests.post(searchurl, |
||||
|
data=payload2.format(self.statenum, self.semester), |
||||
|
cookies=self.cookies, |
||||
|
allow_redirects=False, |
||||
|
headers=custom_headers).content |
||||
|
else: |
||||
|
return first_req |
||||
|
|
||||
|
def classes(self, subject): |
||||
|
return list(parseColumns(subject, self.getlist(subject))) |
||||
|
|
||||
|
def getCodes(self, letter): |
||||
|
sys.stderr.write("Getting letter " + letter + "\n") |
||||
|
first_req = requests.get(searchurl, cookies=self.cookies).content |
||||
|
self.statenum = getStateNum(first_req) |
||||
|
|
||||
|
self.statenum = getStateNum(requests.post(searchurl, |
||||
|
data=courseCodes1.format(self.statenum, self.semester), |
||||
|
cookies=self.cookies, |
||||
|
headers=custom_headers).content) |
||||
|
|
||||
|
return getCodes(requests.post(searchurl, |
||||
|
data=courseCodes2.format(self.statenum, letter, self.semester), |
||||
|
cookies=self.cookies, |
||||
|
allow_redirects=False, |
||||
|
headers=custom_headers).content) |
||||
|
@property |
||||
|
def codes(self): |
||||
|
if not self.codes_: |
||||
|
self.codes_ = list(chain.from_iterable( |
||||
|
map((lambda l: |
||||
|
self.getCodes(chr(l))), |
||||
|
xrange(65, 91)))) |
||||
|
return self.codes_ |
||||
|
|
||||
|
def request(codes, lists, semester): |
||||
|
requester = MosReq(semester) |
||||
|
while not codes.empty(): |
||||
|
code = codes.get() |
||||
|
try: |
||||
|
lists.put(requester.classes(code)) |
||||
|
except: |
||||
|
codes.task_done() |
||||
|
return |
||||
|
codes.task_done() |
||||
|
|
||||
|
|
||||
|
class CourseInfo(object): |
||||
|
def __init__(self, threadcount, semester): |
||||
|
self._codes = False |
||||
|
self.threadcount = threadcount |
||||
|
self.semester = semester |
||||
|
|
||||
|
@property |
||||
|
def codes(self): |
||||
|
if not self._codes: |
||||
|
req = MosReq(self.semester) |
||||
|
self._codes = req.codes |
||||
|
return self._codes |
||||
|
|
||||
|
def classes(self): |
||||
|
qcodes = q.Queue() |
||||
|
for code in self.codes: |
||||
|
qcodes.put(code) |
||||
|
lists = q.Queue() |
||||
|
threads = [] |
||||
|
thread = None |
||||
|
for i in xrange(self.threadcount): |
||||
|
thread = thd.Thread(group=None, target=request, args=(qcodes, lists, self.semester)) |
||||
|
threads.append(thread) |
||||
|
thread.start() |
||||
|
qcodes.join() |
||||
|
for t in threads: |
||||
|
t.join() |
||||
|
|
||||
|
sections = [] |
||||
|
while not lists.empty(): |
||||
|
sections.append(lists.get()) |
||||
|
|
||||
|
for cl in chain.from_iterable(sections): |
||||
|
new_sections = [] |
||||
|
for sec in cl[1]: |
||||
|
if len(sec.day) > 1: |
||||
|
for day in sec.day: |
||||
|
new_sections.append(copy.deepcopy(sec)) |
||||
|
new_sections[-1]._day = day |
||||
|
else: |
||||
|
sec._day = sec.day[0] |
||||
|
new_sections.append(sec) |
||||
|
yield Class(cl[0][0], sub("\xa0+", "", cl[0][1]), sorted(new_sections)) |
||||
|
|
||||
|
def getCourses(semester, threadcount=10): |
||||
|
return CourseInfo(threadcount, semester).classes() |
||||
|
|
||||
|
def allCourses(): |
||||
|
return chain.from_iterable( |
||||
|
(getCourses(sem, threadcount=10) |
||||
|
for sem in (fall, winter, spring_summer))) |
||||
|
|
||||
|
#for course in allCourses(): |
||||
|
#sys.stdout.write("%s, %s, %s, %s\n" % (course.title, course.code, course.dept, course.books)) |
||||
|
#print course.sections |
||||
@ -0,0 +1,9 @@ |
|||||
|
from oersearch import Search |
||||
|
from classes import getCourses |
||||
|
from sylla import getTextbooks |
||||
|
|
||||
|
mcmasterSearch = Search("McMaster") |
||||
|
|
||||
|
mcmasterSearch.setup(getCourses) |
||||
|
|
||||
|
mcmasterSearch.run() |
||||
@ -0,0 +1,117 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
from sys import argv |
||||
|
from itertools import chain, islice, izip_longest, izip as zip |
||||
|
from re import search, sub |
||||
|
from functools import total_ordering |
||||
|
from re import sub |
||||
|
|
||||
|
import datetime as dt |
||||
|
import lxml.html as lxh |
||||
|
import requests |
||||
|
|
||||
|
# Purpose of this module is to download and parse syllabi from various departments |
||||
|
# In order to be corellated with individual courses |
||||
|
|
||||
|
class Price(object): |
||||
|
def __init__(self, amnt, status): |
||||
|
self.dollars = float(amnt[1:]) |
||||
|
self.status = status |
||||
|
|
||||
|
def __repr__(self): |
||||
|
return "$%s %s" % (repr(self.dollars), self.status) |
||||
|
|
||||
|
|
||||
|
class Book(object): |
||||
|
def __init__(self, title, price): |
||||
|
self.title = title |
||||
|
self.price = price |
||||
|
|
||||
|
def __repr__(self): |
||||
|
return '["%s", "%s"]' % (self.title, repr(self.price)) |
||||
|
|
||||
|
|
||||
|
def grouper(n, iterable, fillvalue=None): |
||||
|
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx" |
||||
|
args = [iter(iterable)] * n |
||||
|
return izip_longest(fillvalue=fillvalue, *args) |
||||
|
|
||||
|
searchUrl = "https://campusstore.mcmaster.ca/cgi-mcm/ws/txsub.pl?wsDEPTG1=%s&wsDEPTDESC1=&wsCOURSEG1=%s&crit_cnt=1" |
||||
|
|
||||
|
def normalize(word): |
||||
|
if len(word) > 1: |
||||
|
return ("%s%s" % |
||||
|
(word[0].upper(), |
||||
|
"".join(word[1:]).lower())) |
||||
|
return word |
||||
|
|
||||
|
def parseAuthor(author): |
||||
|
split = author.split(" ") |
||||
|
if len(split) <= 1: |
||||
|
return author |
||||
|
lastname = split[0] |
||||
|
firstname = split[1] |
||||
|
return "%s %s" % (firstname, lastname) |
||||
|
|
||||
|
def normwords(phrase): |
||||
|
words = phrase.split(" ") |
||||
|
return " ".join(map(normalize, words)) |
||||
|
|
||||
|
def books(dept, code, withPrices): |
||||
|
""" |
||||
|
Snatch me up a book title or three |
||||
|
""" |
||||
|
req = searchUrl % (dept, code) |
||||
|
|
||||
|
html = requests.get(req).text |
||||
|
|
||||
|
parsed = lxh.fromstring(html) |
||||
|
|
||||
|
pricelist = prices(parsed) |
||||
|
|
||||
|
for div in parsed.xpath(".//div"): |
||||
|
if (div.attrib.has_key("id") and |
||||
|
"prodDesc" in div.attrib["id"]): |
||||
|
|
||||
|
textbook = div.text_content() |
||||
|
author = sub(r',', '', |
||||
|
"".join( |
||||
|
(div.getparent() |
||||
|
.xpath(".//span[@class='inline']") |
||||
|
[0].text_content() |
||||
|
.split(":")[1:])).strip()) |
||||
|
price = pricelist.pop() |
||||
|
if withPrices: |
||||
|
yield (normwords(textbook), normwords(author), repr(price)) |
||||
|
else: |
||||
|
yield (normwords(textbook), normwords(author)) |
||||
|
|
||||
|
def prices(html): |
||||
|
""" |
||||
|
Get the prices from a search result page |
||||
|
""" |
||||
|
ps = [ |
||||
|
p.getparent().text_content().split()[0] |
||||
|
for p in html.xpath("//p/input[@type='checkbox']") |
||||
|
] |
||||
|
|
||||
|
try: |
||||
|
amts, stats = zip(*list(reversed(list(grouper(2, ps))))) |
||||
|
return map(Price, amts, stats) |
||||
|
except ValueError: |
||||
|
return [] |
||||
|
|
||||
|
def textbookInfo(dept, code, withPrices=False): |
||||
|
""" |
||||
|
Return all the textbooks for a course |
||||
|
""" |
||||
|
return list(books(dept, code, withPrices)) |
||||
|
|
||||
|
def humanities(): |
||||
|
""" |
||||
|
Download humanities syllabi |
||||
|
""" |
||||
|
return [] |
||||
|
|
||||
|
# Example, getting the course info for Personality Theory (PSYCH = Department, 2B03 = Course code) |
||||
|
# print list(courseInfo("PSYCH", "2B03")) |
||||
@ -0,0 +1,24 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
from urllib import quote |
||||
|
from json import loads, dumps |
||||
|
|
||||
|
import requests as req |
||||
|
|
||||
|
#query = "https://openlibrary.org/query.json?type=/type/edition&title=%s&author=%s" |
||||
|
searchurl = 'http://openlibrary.org/search.json?author=%s&title=%s' |
||||
|
|
||||
|
def bookUrls(title, author): |
||||
|
print title, author |
||||
|
if ":" in title: |
||||
|
title = title.split(":")[0] |
||||
|
requrl = searchurl % (quote(author), quote(title)) |
||||
|
results = loads(req.get(requrl).text) |
||||
|
for result in results["docs"][0:2]: |
||||
|
if result.has_key("edition_key"): |
||||
|
yield "https://openlibrary.org/books/%s" % result["edition_key"][0] |
||||
|
|
||||
|
# 'http://openlibrary.org/query.json?type=/type/edition&title=The+Personality+Puzzle' |
||||
|
|
||||
|
#for book in bookUrls("Philosophy Of Physics", "Tim Maudlin"): |
||||
|
#print book |
||||
@ -0,0 +1,153 @@ |
|||||
|
##! /usr/bin/python2 |
||||
|
from itertools import groupby, chain |
||||
|
from sys import stdout |
||||
|
from functools import partial |
||||
|
from json import dumps |
||||
|
|
||||
|
def gensymer(): |
||||
|
n = [0] |
||||
|
def inner(): |
||||
|
result = str(n[0]) |
||||
|
n[0] += 1 |
||||
|
return result |
||||
|
return inner |
||||
|
|
||||
|
gensym = gensymer() |
||||
|
|
||||
|
def printTrie(graph, prev, trie, weight): |
||||
|
new_node = str(gensym()) |
||||
|
graph.node(new_node, "%s" % trie.letter) |
||||
|
graph.edge(prev, new_node, label="%.2f" % weight) |
||||
|
if not trie.children: |
||||
|
return |
||||
|
for child, weight in zip(trie.children, trie.ws): |
||||
|
printTrie(graph, new_node, child, weight) |
||||
|
|
||||
|
|
||||
|
class Trie(object): |
||||
|
def __init__(self, letter, children, ws): |
||||
|
self.letter = letter |
||||
|
self.children = children |
||||
|
self.ws = ws |
||||
|
|
||||
|
def probweight(suffixes): |
||||
|
weights = [float(s["value"]) for s in suffixes] |
||||
|
s = float(sum(weights)) |
||||
|
ws = [w/s for w in weights] |
||||
|
return ws |
||||
|
|
||||
|
def buildtrie(trie, suffixes): |
||||
|
""" |
||||
|
Build a trie, also known as a prefix tree, of all the possible completions |
||||
|
""" |
||||
|
trie.children = [] |
||||
|
for letter, suffs in suffixes: |
||||
|
ped = partition(suffs) |
||||
|
if any(map(lambda p: p[0], ped)): |
||||
|
# check if there are any children |
||||
|
trie.children.append(buildtrie(Trie(letter, [], probweight(suffs)), partition(suffs))) |
||||
|
else: |
||||
|
# we've reached the end of this word so just include the final letter |
||||
|
# [1] = there is a probability of 1 of reaching this single leaf node, |
||||
|
# since it is the only possible completion here |
||||
|
trie.children.append(Trie(letter, [], [1])) |
||||
|
return trie |
||||
|
|
||||
|
|
||||
|
def keyf(x): |
||||
|
if not x["key"]: |
||||
|
return "" |
||||
|
return x["key"][0] |
||||
|
|
||||
|
def tails(words): |
||||
|
for word in words: |
||||
|
yield { |
||||
|
"key" : word["key"][1:], |
||||
|
"value" : word["value"] |
||||
|
} |
||||
|
|
||||
|
def partition(words): |
||||
|
""" |
||||
|
Partition the words into different prefixes based on the first character |
||||
|
""" |
||||
|
groups = [ |
||||
|
(g[0], list(tails(g[1]))) |
||||
|
for g in groupby( |
||||
|
sorted(words, key=keyf), |
||||
|
key=keyf) |
||||
|
] |
||||
|
return groups |
||||
|
|
||||
|
|
||||
|
def flatten_helper(letter, trie): |
||||
|
return ([letter + child.letter for |
||||
|
child in trie.children], trie.children) |
||||
|
|
||||
|
def flatten(trie): |
||||
|
if not trie.children: |
||||
|
return trie.letter |
||||
|
prefixes, suffixes = flatten_helper(trie.letter, trie) |
||||
|
return [flatten(Trie(p, s2.children, s2.ws)) for p, s2 in zip(prefixes, suffixes)] |
||||
|
|
||||
|
def flattenlist(xs): |
||||
|
locs = [] |
||||
|
for x in xs: |
||||
|
if not isinstance(x, list): |
||||
|
locs.append(x) |
||||
|
else: |
||||
|
locs.extend(flattenlist(x)) |
||||
|
return locs |
||||
|
|
||||
|
def matchc(trie, prefix): |
||||
|
c = None |
||||
|
if len(prefix) > 1: |
||||
|
c = prefix[0] |
||||
|
else: |
||||
|
c = prefix |
||||
|
return [ch for ch in trie.children if ch.letter == c] |
||||
|
|
||||
|
def match(trie, word): |
||||
|
if not word: |
||||
|
return [] |
||||
|
m = matchc(trie, word[0]) |
||||
|
if not m: |
||||
|
return [] |
||||
|
else: |
||||
|
return [m[0]] + match(m[0], word[1:]) |
||||
|
|
||||
|
def complete(trie, word): |
||||
|
m = match(trie, word) |
||||
|
if len(word) != len(m): |
||||
|
return False |
||||
|
completions = [word+x[1:] for x in flattenlist(flatten(m[-1]))] |
||||
|
if len(completions) > 10: |
||||
|
return dumps(completions[0:10]) |
||||
|
return dumps(completions) |
||||
|
|
||||
|
def sortTrie(trie): |
||||
|
""" |
||||
|
Sort the children of each node in descending order |
||||
|
of the probability that each child would be the completion |
||||
|
of whatever that word is |
||||
|
""" |
||||
|
if not trie.children: |
||||
|
return |
||||
|
sortedChilds = sorted(zip(trie.children, trie.ws), key=lambda x: x[1], reverse=True) |
||||
|
trie.children = [x[0] for x in sortedChilds] |
||||
|
trie.ws = [x[1] for x in sortedChilds] |
||||
|
for child in trie.children: |
||||
|
sortTrie(child) |
||||
|
|
||||
|
def toTrie(words): |
||||
|
for word in words: |
||||
|
word["key"] = word["key"].lower() |
||||
|
trie = buildtrie(Trie("", [], [1]), partition(words)) |
||||
|
trie.ws = [1]*len(trie.children) |
||||
|
sortTrie(trie) |
||||
|
return trie |
||||
|
|
||||
|
def testkey(w): |
||||
|
return { |
||||
|
"key" : w, |
||||
|
"value" : "1" |
||||
|
} |
||||
@ -0,0 +1,67 @@ |
|||||
|
#lang racket |
||||
|
|
||||
|
(require json) |
||||
|
|
||||
|
(define (root name type) |
||||
|
`(,name type)) |
||||
|
|
||||
|
(define ((prop |
||||
|
type |
||||
|
[index "analyzed"]) |
||||
|
name) |
||||
|
(define prop-hash (make-hash)) |
||||
|
(hash-set! prop-hash (string->symbol name) |
||||
|
`#hash( |
||||
|
(type . ,type) |
||||
|
(index . ,index))) |
||||
|
prop-hash) |
||||
|
|
||||
|
(define ((dict name) ps) |
||||
|
(let ([prop-vals (make-hash)] |
||||
|
[props-hsh (make-hash)]) |
||||
|
(for ([p ps]) |
||||
|
(hash-set! prop-vals |
||||
|
(car p) |
||||
|
(cdr p))) |
||||
|
(hash-set! props-hsh name prop-vals) |
||||
|
props-hsh)) |
||||
|
|
||||
|
(define (props ps) |
||||
|
(define props-hash (make-hash)) |
||||
|
(for ([p ps]) |
||||
|
(match p |
||||
|
[(hash-table (k v)) |
||||
|
(hash-set! props-hash k v)])) |
||||
|
`#hash((properties . ,props-hash))) |
||||
|
|
||||
|
(define str (prop "string")) |
||||
|
|
||||
|
(define num (prop "integer")) |
||||
|
|
||||
|
(define date (prop "date")) |
||||
|
|
||||
|
(define bool (prop "boolean" "not_analyzed")) |
||||
|
|
||||
|
(define (dictprop name d) |
||||
|
(define dict (make-hash)) |
||||
|
(hash-set! dict name d) |
||||
|
dict) |
||||
|
|
||||
|
(define (estruct name pairs) |
||||
|
(define estr (make-hash)) |
||||
|
(hash-set! estr |
||||
|
(string->symbol name) |
||||
|
(props pairs)) |
||||
|
estr) |
||||
|
|
||||
|
(define (make-mapping |
||||
|
type |
||||
|
decl) |
||||
|
(define mapping (make-hash)) |
||||
|
(hash-set! mapping |
||||
|
(string->symbol type) |
||||
|
(props decl)) |
||||
|
(jsexpr->string mapping)) |
||||
|
|
||||
|
(provide |
||||
|
(all-defined-out)) |
||||
@ -0,0 +1,35 @@ |
|||||
|
<book> |
||||
|
<div class="text-clip toast" if={ opts.title && opts.author }> |
||||
|
<p> |
||||
|
<dt class="book-title text-center"> |
||||
|
<button onclick={ makeResourceGetter(this) } class="btn btn-link"> |
||||
|
{ opts.title } |
||||
|
</button> |
||||
|
</dt> |
||||
|
<dd> |
||||
|
<div if={ loading } class="loading"> |
||||
|
</div> |
||||
|
<p if={ iarchive }> |
||||
|
<a target="_blank" href="{ iarchive }"> |
||||
|
<button class="centered btn btn-link"> |
||||
|
Internet Archive Result |
||||
|
</button> |
||||
|
</a> |
||||
|
</p> |
||||
|
<p if={ openlib }> |
||||
|
<a target="_blank" href="{ openlib }"> |
||||
|
<button class="centered btn btn-link"> |
||||
|
Open Library Result |
||||
|
</button> |
||||
|
</a> |
||||
|
</p> |
||||
|
<p class="centered" if={ noResources }> |
||||
|
Couldn't find anything, sorry :( |
||||
|
</p> |
||||
|
</dd> |
||||
|
</p> |
||||
|
</div> |
||||
|
this.iarchive = false; |
||||
|
this.openlib = false; |
||||
|
this.noResources = false; |
||||
|
</book> |
||||
@ -0,0 +1,32 @@ |
|||||
|
<class> |
||||
|
<div class="card-header"> |
||||
|
<div id='title'> { dept } { title } </div> |
||||
|
<div id='prof'> { prof } </div> |
||||
|
<div id='sem'> { sem } </div> |
||||
|
</div> |
||||
|
<div if={ books } class="card-body"> |
||||
|
<button onclick={ makeShow(this) } class="btn btn-primary show-button"> |
||||
|
<strong>Show Textbooks</strong> |
||||
|
</button> |
||||
|
<div if={ showBooks }> |
||||
|
<dl> |
||||
|
<book each={ books } |
||||
|
data="{ this }" |
||||
|
resources="" |
||||
|
title={ booktitle } |
||||
|
author={ bookauthor } |
||||
|
price={ bookprice }> |
||||
|
</book> |
||||
|
</dl> |
||||
|
</div> |
||||
|
</div> |
||||
|
<div class="toast" if={ !books }> |
||||
|
<p>No books at this time</p> |
||||
|
<p>Check back later, or verify the course has books</p> |
||||
|
</div> |
||||
|
</class> |
||||
|
|
||||
|
<script> |
||||
|
this.showBooks = false; |
||||
|
this.update(); |
||||
|
</script> |
||||
@ -0,0 +1,13 @@ |
|||||
|
<results> |
||||
|
<div class="courses container"> |
||||
|
<row class="columns" each={ rows } data="{ this }" classrow={ row }></row> |
||||
|
</div> |
||||
|
this.rows = []; |
||||
|
var self = this; |
||||
|
results_passer.on("new_results", |
||||
|
function(data) { |
||||
|
console.log("new search results detected"); |
||||
|
self.rows = data; |
||||
|
self.update(); |
||||
|
}); |
||||
|
</results> |
||||
@ -0,0 +1,6 @@ |
|||||
|
<row> |
||||
|
<class class="course text-ellipsis text-justify rounded card column col-md-3" each="{ classrow }" data="{ this }"></class> |
||||
|
|
||||
|
this.classrow = opts.classrow |
||||
|
</row> |
||||
|
|
||||
@ -0,0 +1,96 @@ |
|||||
|
function makeResourceGetter(self) { |
||||
|
function getResources(ev) { |
||||
|
ev.preventDefault(); |
||||
|
self.loading = true; |
||||
|
self.update(); |
||||
|
var params = { |
||||
|
"title" : this.booktitle, |
||||
|
"author" : this.bookauthor |
||||
|
}; |
||||
|
var url = "http://localhost:8001/resources"; |
||||
|
console.log(params); |
||||
|
$.getJSON(url, { |
||||
|
data : JSON.stringify(params) |
||||
|
}).done(function(results) { |
||||
|
|
||||
|
if (results.iarchive) { |
||||
|
self.iarchive = results.iarchive[0]; |
||||
|
} |
||||
|
|
||||
|
if (results.openlib) { |
||||
|
self.openlib = results.openlib[0]; |
||||
|
} |
||||
|
|
||||
|
if (!(results.openlib && results.iarchive)) { |
||||
|
self.noResources = true; |
||||
|
} |
||||
|
self.loading = false; |
||||
|
self.update(); |
||||
|
}); |
||||
|
} |
||||
|
return getResources; |
||||
|
} |
||||
|
|
||||
|
function makeShow(self) { |
||||
|
return function() { |
||||
|
if (!self.showBooks) { |
||||
|
self.showBooks = true; |
||||
|
} |
||||
|
else { |
||||
|
self.showBooks = false; |
||||
|
} |
||||
|
self.update(); |
||||
|
}; |
||||
|
} |
||||
|
|
||||
|
function ResultsPasser() { |
||||
|
riot.observable(this); |
||||
|
return this; |
||||
|
} |
||||
|
|
||||
|
var results_passer = new ResultsPasser(); |
||||
|
|
||||
|
riot.mount("search"); |
||||
|
riot.mount("results"); |
||||
|
|
||||
|
function autocomplete(element, endpoint) { |
||||
|
// The element should be an input class
|
||||
|
$(element).autocomplete({ |
||||
|
source : endpoint, |
||||
|
my : "right top", |
||||
|
at : "left bottom", |
||||
|
collision : "none", |
||||
|
autofocus : true, |
||||
|
delay : 100 |
||||
|
}); |
||||
|
} |
||||
|
|
||||
|
function filterCourses(courses) { |
||||
|
return courses.filter( |
||||
|
function (c) { |
||||
|
return c.prof != "Staff"; |
||||
|
}); |
||||
|
} |
||||
|
|
||||
|
function take(n, xs) { |
||||
|
return xs.slice(0, n); |
||||
|
} |
||||
|
|
||||
|
function drop(n, xs) { |
||||
|
return xs.slice(n, xs.length); |
||||
|
} |
||||
|
|
||||
|
function groupsof(n, xs) { |
||||
|
var groups = []; |
||||
|
while (xs.length != 0) { |
||||
|
if (xs.length < n) { |
||||
|
groups.push({"row" : take(xs.length, xs)} ); |
||||
|
xs = drop(xs.length, xs); |
||||
|
} |
||||
|
else { |
||||
|
groups.push({"row" : take(n, xs)}); |
||||
|
xs = drop(n, xs); |
||||
|
} |
||||
|
} |
||||
|
return groups; |
||||
|
} |
||||
@ -0,0 +1,30 @@ |
|||||
|
<search> |
||||
|
<form class="form-horizontal search-form" onsubmit={ submit } type="submit"method="get"> |
||||
|
<div class="form-group"> |
||||
|
<div class="col-sm-8 form-item"> |
||||
|
<input class="form-input" placeholder="Description" type="text" name="title"/> |
||||
|
</div> |
||||
|
<div class="col-sm-2 form-item"> |
||||
|
<select class="form-select" aria-labelledby="dLabel" name="sem"> |
||||
|
<option value="Fall">Fall</option> |
||||
|
<option value="Winter" selected>Winter</option> |
||||
|
<option value="Summer">Summer</option> |
||||
|
</select> |
||||
|
</div> |
||||
|
<div class="col-sm-2 form-item"> |
||||
|
<button class="btn btn-primary" type="submit">Search</button> |
||||
|
</div> |
||||
|
</div> |
||||
|
</form> |
||||
|
</search> |
||||
|
|
||||
|
function submit(ev) { |
||||
|
console.log("submitted"); |
||||
|
var params = $(ev.currentTarget).serialize(); |
||||
|
$.getJSON("http://localhost:8001/fc?"+params, |
||||
|
function(courses) { |
||||
|
var fcourses = filterCourses(courses); |
||||
|
var cgroups = groupsof(4, fcourses); |
||||
|
results_passer.trigger("new_results", cgroups); |
||||
|
}); |
||||
|
} |
||||
@ -0,0 +1,237 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
import elasticsearch |
||||
|
|
||||
|
from elasticsearch_dsl import FacetedSearch, Search, Q |
||||
|
from elasticsearch_dsl.aggs import Terms, DateHistogram |
||||
|
from sys import exit, stderr |
||||
|
from json import dumps, loads |
||||
|
from itertools import chain, imap |
||||
|
|
||||
|
from hashlib import sha1 |
||||
|
|
||||
|
from textbookExceptions import UnIndexable |
||||
|
|
||||
|
from mcmaster.classes import allCourses |
||||
|
|
||||
|
# Generic instance of elasticsearch right now |
||||
|
es = elasticsearch.Elasticsearch() |
||||
|
|
||||
|
def summarize(text): |
||||
|
splitted = text.split(" ") |
||||
|
if len(splitted) > 4: |
||||
|
return " ".join(splitted[0:4]) + ".." |
||||
|
return text |
||||
|
|
||||
|
def sectionToJSON(section): |
||||
|
return { |
||||
|
"prof" : section.prof, |
||||
|
"sem" : section.sem, |
||||
|
"day" : section.day |
||||
|
} |
||||
|
|
||||
|
def classToJSON(clss): |
||||
|
return { |
||||
|
"title" : clss.title, |
||||
|
"sections" : map(sectionToJSON, clss.sections), |
||||
|
"dept" : clss.dept, |
||||
|
"code" : clss.code, |
||||
|
"books" : list(clss.books) if clss.books else [] |
||||
|
} |
||||
|
|
||||
|
|
||||
|
def truncate(docid): |
||||
|
""" |
||||
|
Truncate a document id to 12 digits |
||||
|
The document ID should be based on a |
||||
|
hash of unique identifiers |
||||
|
""" |
||||
|
return int(str(docid)[0:12]) |
||||
|
|
||||
|
def hashsec(course): |
||||
|
""" |
||||
|
Hash a course into a usable id |
||||
|
""" |
||||
|
if not course["code"]: |
||||
|
code = "" |
||||
|
else: |
||||
|
code = course["code"] |
||||
|
if not course["title"]: |
||||
|
title = "" |
||||
|
else: |
||||
|
title = course["title"] |
||||
|
|
||||
|
if not course["sections"] or len(course["sections"]) < 1: |
||||
|
course["sections"][0] = "" |
||||
|
|
||||
|
if not (code or title): |
||||
|
raise UnIndexable(course) |
||||
|
|
||||
|
h = sha1() |
||||
|
h.update(code + title + course["sections"][0]["sem"]) |
||||
|
return int(h.hexdigest(), 16) |
||||
|
|
||||
|
def createIndex(name): |
||||
|
""" |
||||
|
This creates a new index in elasticsearch |
||||
|
An index is like a schema in a regular database |
||||
|
Create an elasticsearch index |
||||
|
|
||||
|
""" |
||||
|
indices = elasticsearch.client.IndicesClient(es) |
||||
|
|
||||
|
print indices.create(name) |
||||
|
with open("./course.json", "r") as mapping: |
||||
|
print indices.put_mapping("course", loads(mapping.read()), name) |
||||
|
|
||||
|
def indexListing(course): |
||||
|
""" |
||||
|
Index a specific course in the database (using the courses index) |
||||
|
example, |
||||
|
{ |
||||
|
'books': [], |
||||
|
'dept': 'COLLAB', |
||||
|
'code': '2C03', |
||||
|
'sections': [ |
||||
|
{ |
||||
|
'prof': 'Lisa Pender', |
||||
|
'sem': '2015/09/08 - 2015/12/08', |
||||
|
'day': 'Mo' |
||||
|
}, |
||||
|
{ |
||||
|
'prof': 'Staff', |
||||
|
'sem': '2015/09/08 - 2015/12/08', |
||||
|
'day': 'Th' |
||||
|
} |
||||
|
], |
||||
|
'title': 'COLLAB 2C03 - Sociology I' |
||||
|
} |
||||
|
|
||||
|
""" |
||||
|
courseID = hashsec(course) |
||||
|
print es.index(index="oersearch", |
||||
|
doc_type="course", |
||||
|
id=courseID, |
||||
|
body=course) |
||||
|
|
||||
|
# For every course we index, we also create a resource for it |
||||
|
# This should be an idempotent operation because we're putting it in couchdb |
||||
|
# And we're using the id obtained from the hash function, so it should just update the document |
||||
|
# no need to delete anything |
||||
|
#try: |
||||
|
#courseDept = course[0]["title"].strip().split(" ")[0].strip() |
||||
|
#courseCode = course[0]["title"].strip().split(" ")[1].strip() |
||||
|
#print "DEPARTMENT = \"%s\", COURSECODE = \"%s\"" % (courseDept, courseCode) |
||||
|
#print createResource(textbookInfo, course[0], courseDept, courseCode, courseID) |
||||
|
#except: |
||||
|
#print "Couldn't create the resource associated with %s" % course |
||||
|
|
||||
|
def termSearch(field): |
||||
|
""" |
||||
|
Make a term search (exact match) |
||||
|
""" |
||||
|
def t(term): |
||||
|
q = Q("term", |
||||
|
**{ |
||||
|
"sections."+field : term |
||||
|
}) |
||||
|
return q |
||||
|
return t |
||||
|
|
||||
|
def search(field): |
||||
|
""" |
||||
|
Make a match search |
||||
|
""" |
||||
|
def s(term): |
||||
|
q = Q("match", |
||||
|
**{ |
||||
|
field : term |
||||
|
}) |
||||
|
return q |
||||
|
return s |
||||
|
|
||||
|
def join(x, y): |
||||
|
""" |
||||
|
Join two queries |
||||
|
""" |
||||
|
return x & y |
||||
|
|
||||
|
def filterSections(secs): |
||||
|
""" |
||||
|
Get rid of tutorial sections |
||||
|
because they almost always have "Staff" as the instructor |
||||
|
This is just a heuristic of course |
||||
|
""" |
||||
|
filtered = [s for s in secs.sections if "Staff" not in s.prof] |
||||
|
if len(filtered) > 0: |
||||
|
return filtered |
||||
|
return False |
||||
|
|
||||
|
def searchTerms(terms): |
||||
|
""" |
||||
|
Run a search for courses |
||||
|
""" |
||||
|
|
||||
|
# A list of all the queries we want to run |
||||
|
qs = [searchers[field](term) for |
||||
|
field, term in |
||||
|
terms.iteritems() if |
||||
|
term and searchers.has_key(field)] |
||||
|
|
||||
|
if not qs: |
||||
|
# No queries = no results |
||||
|
return dumps([]) |
||||
|
|
||||
|
# Reduce joins all of the queries into one query |
||||
|
# It will search for the conjunction of all of them |
||||
|
# So that means it cares about each query equally |
||||
|
q = reduce(join, qs) |
||||
|
|
||||
|
s = (Search(using=es, index="oersearch") |
||||
|
.query(q))[0:100] # only return up to 100 results for now |
||||
|
|
||||
|
results = s.execute() |
||||
|
|
||||
|
filtered = [ |
||||
|
(secs, filterSections(secs)[0].to_dict()) # get rid of tutorials |
||||
|
for secs in results |
||||
|
if filterSections(secs) |
||||
|
] |
||||
|
results = [] |
||||
|
for obj, secs in filtered: |
||||
|
# Add the truncated course id |
||||
|
# This is used to point to the resource page for that course |
||||
|
secs["id"] = truncate(obj.meta.id) |
||||
|
secs["title"] = obj.title |
||||
|
if obj["dept"] not in secs["title"]: |
||||
|
secs["dept"] = obj.dept |
||||
|
if obj.books: |
||||
|
secs["books"] = [ |
||||
|
{ |
||||
|
"booktitle" : summarize(book[0].encode("ASCII")), |
||||
|
"bookauthor" : book[1].encode("ASCII"), |
||||
|
"bookprice" : book[2].encode("ASCII") |
||||
|
} |
||||
|
for book in obj.books |
||||
|
] |
||||
|
else: |
||||
|
secs["books"] = "" |
||||
|
results.append(secs) |
||||
|
|
||||
|
return dumps(results) |
||||
|
|
||||
|
|
||||
|
searchers = { |
||||
|
"title" : search("title"), |
||||
|
"loc" : search("loc"), |
||||
|
"time" : search("time"), |
||||
|
"prof" : search("prof"), |
||||
|
"day" : search("day"), |
||||
|
} |
||||
|
|
||||
|
#print searchTerms({"title" : "PHILOS"}) |
||||
|
|
||||
|
#for c in imap(classToJSON, allCourses()): |
||||
|
#try: |
||||
|
#print indexListing(c) |
||||
|
#except UnIndexable as e: |
||||
@ -0,0 +1,113 @@ |
|||||
|
header { |
||||
|
color: #1c75bc; |
||||
|
} |
||||
|
|
||||
|
.body { |
||||
|
color: #1c75bc; |
||||
|
} |
||||
|
|
||||
|
a { |
||||
|
color: #1c75bc !important; |
||||
|
} |
||||
|
|
||||
|
.btn-primary { |
||||
|
background-color: #1c75bc !important; |
||||
|
} |
||||
|
|
||||
|
.courses { |
||||
|
margin-top: 100px; |
||||
|
max-width: 85%; |
||||
|
} |
||||
|
|
||||
|
.course { |
||||
|
margin-left: 5px !important; |
||||
|
margin-right: 5px !important; |
||||
|
margin-top: 5px !important; |
||||
|
margin-bottom: 5px !important; |
||||
|
} |
||||
|
|
||||
|
.search-form { |
||||
|
margin-top: 5%; |
||||
|
-webkit-appearance: none !important; |
||||
|
} |
||||
|
|
||||
|
.book-title { |
||||
|
margin-right: 10px !important; |
||||
|
} |
||||
|
|
||||
|
.form-item { |
||||
|
margin-left: 15px; |
||||
|
margin-right 15px; |
||||
|
} |
||||
|
|
||||
|
#title { |
||||
|
font-weight: bolder; |
||||
|
} |
||||
|
|
||||
|
.ui-autocomplete { |
||||
|
position: absolute; |
||||
|
top: 100%; |
||||
|
left: 0; |
||||
|
z-index: 1000; |
||||
|
float: left; |
||||
|
display: none; |
||||
|
min-width: 160px; |
||||
|
padding: 4px 0; |
||||
|
margin: 0 0 10px 25px; |
||||
|
list-style: none; |
||||
|
background-color: #ffffff; |
||||
|
border-color: #ccc; |
||||
|
border-color: rgba(0, 0, 0, 0.2); |
||||
|
border-style: solid; |
||||
|
border-width: 1px; |
||||
|
-webkit-border-radius: 5px; |
||||
|
border-radius: 5px; |
||||
|
-webkit-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2); |
||||
|
box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2); |
||||
|
-moz-background-clip: padding; |
||||
|
background-clip: padding-box; |
||||
|
} |
||||
|
|
||||
|
.ui-menu-item > a.ui-corner-all { |
||||
|
display: block; |
||||
|
padding: 3px 15px; |
||||
|
clear: both; |
||||
|
font-weight: normal; |
||||
|
line-height: 18px; |
||||
|
color: #555555; |
||||
|
white-space: nowrap; |
||||
|
text-decoration: none; |
||||
|
} |
||||
|
|
||||
|
.ui-state-hover, .ui-state-active { |
||||
|
color: #ffffff; |
||||
|
text-decoration: none; |
||||
|
background-color: #0088cc; |
||||
|
border-radius: 0px; |
||||
|
-webkit-border-radius: 0px; |
||||
|
background-image: none; |
||||
|
} |
||||
|
|
||||
|
.logo-div { |
||||
|
height:67px; |
||||
|
width:150px; |
||||
|
margin-right:1%; |
||||
|
margin-bottom: 0%; |
||||
|
margin-top:1%; |
||||
|
margin-left:2%; |
||||
|
background-size: 100%; |
||||
|
background-size: cover; |
||||
|
-webkit-background-size: cover; |
||||
|
-o-background-size: cover; |
||||
|
background-size: cover; |
||||
|
background-position: center center; |
||||
|
/*background-image: url('https://mgoal.ca/goal_transp.png');*/ |
||||
|
} |
||||
|
|
||||
|
.title-div { |
||||
|
margin-right:1%; |
||||
|
margin-bottom: -3%; |
||||
|
margin-top:1%; |
||||
|
margin-left:30%; |
||||
|
font-size: 25px; |
||||
|
} |
||||
File diff suppressed because one or more lines are too long
@ -0,0 +1,47 @@ |
|||||
|
{% extends "bootstrap/base.html" %} |
||||
|
{% block head %} |
||||
|
{{super()}} |
||||
|
<meta name="viewport" content="width=device-width, initial-scale=1"> |
||||
|
<header class="nav navbar"> |
||||
|
<section class="navbar-section"> |
||||
|
<div class="title-div"> |
||||
|
<strong>Search For Books - by the |
||||
|
<a href="http://localhost:8001/blog" target="_blank">Guerilla Open Access League</a> |
||||
|
</strong> |
||||
|
</div> |
||||
|
</section> |
||||
|
<section class="navbar-section"> |
||||
|
<search class="search-form"></search> |
||||
|
</section> |
||||
|
</header> |
||||
|
{% endblock %} |
||||
|
<html> |
||||
|
<body> |
||||
|
{% block content %} |
||||
|
|
||||
|
<results></results> |
||||
|
|
||||
|
{% endblock %} |
||||
|
|
||||
|
<footer class="footer"> |
||||
|
</footer> |
||||
|
|
||||
|
{% block styles %} |
||||
|
{{super()}} |
||||
|
<link rel="stylesheet" href="http://localhost:8001/styles/spectre.min.css"> |
||||
|
<link rel="stylesheet" href="http://localhost:8001/styles/search.css"> |
||||
|
{% endblock %} |
||||
|
|
||||
|
{% block scripts %} |
||||
|
{{super()}} |
||||
|
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/riot/2.4.1/riot+compiler.min.js"></script> |
||||
|
<script type="text/javascript" src="http://localhost:8001/jquery-ui-1.11.4.custom/jquery-ui.min.js"></script> |
||||
|
<script type="riot/tag" src="http://localhost:8001/scripts/search.tag"></script> |
||||
|
<script type="riot/tag" src="http://localhost:8001/scripts/book.tag"></script> |
||||
|
<script type="riot/tag" src="http://localhost:8001/scripts/class.tag"></script> |
||||
|
<script type="riot/tag" src="http://localhost:8001/scripts/row.tag"></script> |
||||
|
<script type="riot/tag" src="http://localhost:8001/scripts/results.tag"></script> |
||||
|
<script type="text/javascript" src="http://localhost:8001/scripts/search.js"></script> |
||||
|
{% endblock %} |
||||
|
</body> |
||||
|
</html> |
||||
@ -0,0 +1,24 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
class UnIndexable(Exception): |
||||
|
def __init__(self, course): |
||||
|
self.course = course |
||||
|
|
||||
|
@property |
||||
|
def reason(self): |
||||
|
course = self.course |
||||
|
if not course["code"] and not course["title"]: |
||||
|
message = "there was no course code and no title defined" |
||||
|
if not course["code"]: |
||||
|
message = "there was no course code defined" |
||||
|
if not course["title"]: |
||||
|
message = "there was no course title defined" |
||||
|
if not course["sections"]: |
||||
|
message = "there were no sections defined" |
||||
|
return """ |
||||
|
There was a problem with indexing this course. |
||||
|
%s |
||||
|
There could be several reasons why, my best guess is that %s |
||||
|
We need at least the course code, title, and one or more sections to index |
||||
|
|
||||
|
""" % (course, message) |
||||
@ -0,0 +1,97 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
|
||||
|
from json import loads, load |
||||
|
from re import sub, split |
||||
|
from itertools import groupby |
||||
|
from numpy import mean |
||||
|
from operator import attrgetter |
||||
|
|
||||
|
import pygal |
||||
|
import csv |
||||
|
|
||||
|
class Textbook(object): |
||||
|
def __init__(self, dept, code, title, author, price): |
||||
|
self.dept = dept |
||||
|
self.code = code |
||||
|
self.title = title |
||||
|
self.author = author |
||||
|
self.price = float(price) |
||||
|
|
||||
|
def __repr__(self): |
||||
|
return "Dept = %s, Code = %s, %s by %s, costs $%s" % (self.dept, |
||||
|
self.code, |
||||
|
self.title, |
||||
|
self.author, |
||||
|
self.price) |
||||
|
|
||||
|
|
||||
|
def courses(): |
||||
|
with open("./books.csv", "r") as books: |
||||
|
booksreader = csv.reader(books) |
||||
|
for row in booksreader: |
||||
|
yield row |
||||
|
|
||||
|
|
||||
|
def groupDept(courselist): |
||||
|
sortedCourses = sorted(courselist, key=attrgetter("dept")) |
||||
|
for course in groupby(sortedCourses, attrgetter("dept")): |
||||
|
yield course[0], list(course[1]) |
||||
|
|
||||
|
def meanPrice(books): |
||||
|
return mean([book.price for book in books]) |
||||
|
|
||||
|
# Questions, |
||||
|
# mean cost per department |
||||
|
# mean cost per faculty |
||||
|
# mean difference between book store copies and other copies per dept and faculty |
||||
|
# number of overlapping books per faculty, do eng students benefit from that? |
||||
|
|
||||
|
# maybe a survey for students to see how often they buy books from other sources |
||||
|
# correlate with how much they could be saving? |
||||
|
|
||||
|
facultyDesc = { |
||||
|
"hum" : "Humanities", |
||||
|
"bus" : "Business", |
||||
|
"hlth" : "Health Science", |
||||
|
"eng" : "Engineering", |
||||
|
"sci" : "Science", |
||||
|
"socsci" : "Social Sciences", |
||||
|
"artsci" : "Arts & Sciences", |
||||
|
"meld" : "MELD" |
||||
|
} |
||||
|
|
||||
|
faculties = load(open("./faculties.json")) |
||||
|
|
||||
|
def categorize(dept): |
||||
|
# faculties |
||||
|
return facultyDesc.get(faculties.get(dept, False), False) |
||||
|
|
||||
|
def byFaculty(): |
||||
|
for dept, books in groupDept(courses()): |
||||
|
yield (categorize(dept), dept, books) |
||||
|
|
||||
|
def meanFacultyCosts(): |
||||
|
byfac = list(byFaculty()) |
||||
|
graph = pygal.Bar() |
||||
|
graph.title = "Mean textbook cost by faculty" |
||||
|
sortedFacs = sorted(byfac, key=lambda x: x[0]) |
||||
|
for fac in groupby(sortedFacs, lambda x: x[0]): |
||||
|
graph.add(fac[0], meanPrice(list(fac[1])[0][2])) |
||||
|
graph.value_formatter = lambda x: '$%.2f' % x if x is not None else "None" |
||||
|
return graph.render(transpose=True) |
||||
|
|
||||
|
def meanCosts(): |
||||
|
cs = groupDept(courses()) |
||||
|
graph = pygal.Bar() |
||||
|
graph.title = "Mean textbook cost by department" |
||||
|
for c in cs: |
||||
|
dept, books = c |
||||
|
graph.add(dept, meanPrice(books)) |
||||
|
#graph.render_to_file("./test_graph.svg") |
||||
|
graph.value_formatter = lambda x: '$%.2f' % x if x is not None else "None" |
||||
|
return graph.render_table(style=True, transpose=True) |
||||
|
|
||||
|
for x in courses(): |
||||
|
print x |
||||
|
#print meanCosts() |
||||
|
#print meanFacultyCosts() |
||||
@ -0,0 +1,148 @@ |
|||||
|
#! /usr/bin/python2 |
||||
|
from functools import partial |
||||
|
from couchdb import ResourceConflict |
||||
|
|
||||
|
from flask import Flask, render_template, flash, request, send_from_directory |
||||
|
from flask_bootstrap import Bootstrap |
||||
|
from flask_appconfig import AppConfig |
||||
|
from urllib import unquote |
||||
|
from search import searchTerms |
||||
|
|
||||
|
from openlibrary import bookUrls |
||||
|
|
||||
|
from archive import searchIA |
||||
|
from urllib import quote, unquote |
||||
|
from json import dumps, loads |
||||
|
|
||||
|
from werkzeug.contrib.cache import MemcachedCache |
||||
|
cache = MemcachedCache(['127.0.0.1:11211']) |
||||
|
|
||||
|
import os |
||||
|
|
||||
|
def predict(fieldtype, term): |
||||
|
print fieldtype |
||||
|
print term |
||||
|
if not term: |
||||
|
return "[]" |
||||
|
else: |
||||
|
try: |
||||
|
cs = completers[fieldtype](term.lower()) |
||||
|
except KeyError: |
||||
|
return "[]" |
||||
|
if cs: |
||||
|
return cs |
||||
|
return "[]" |
||||
|
|
||||
|
def predictor(fieldtype): |
||||
|
def inner(request): |
||||
|
params = dict(request.args.items()) |
||||
|
return predict(fieldtype, params["term"]) |
||||
|
return inner |
||||
|
|
||||
|
def cacheit(key, thunk): |
||||
|
""" |
||||
|
Tries to find a cached version of ``key'' |
||||
|
If there is no cached version then it will |
||||
|
evaluate thunk (which must be a generator) |
||||
|
and cache that, then return the result |
||||
|
""" |
||||
|
cached = cache.get(quote(key)) |
||||
|
if cached is None: |
||||
|
result = list(thunk()) |
||||
|
cache.set(quote(key), result) |
||||
|
return result |
||||
|
return cached |
||||
|
|
||||
|
def ClassSearch(configfile=None): |
||||
|
defaults = {"Day", "Building", "Exact Location", "Department"} |
||||
|
app = Flask(__name__) |
||||
|
AppConfig(app, configfile) # Flask-Appconfig is not necessary, but |
||||
|
# highly recommend =) |
||||
|
# https://github.com/mbr/flask-appconfig |
||||
|
Bootstrap(app) |
||||
|
|
||||
|
app.config["scripts"] = "/home/wes/MGOAL/scripts" |
||||
|
app.config["styles"] = "/home/wes/MGOAL/styles" |
||||
|
|
||||
|
@app.route('/favicon.ico') |
||||
|
def favicon(): |
||||
|
return send_from_directory("/srv/http/goal/favicon.ico", |
||||
|
'favicon.ico', mimetype='image/vnd.microsoft.icon') |
||||
|
|
||||
|
|
||||
|
@app.route("/buildpred", methods=("GET", "POST")) |
||||
|
def buildpred(): |
||||
|
return predictbuild(request) |
||||
|
|
||||
|
@app.route("/locpred", methods=("GET", "POST")) |
||||
|
def locpred(): |
||||
|
return predictloc(request) |
||||
|
|
||||
|
@app.route("/daypred", methods=("GET", "POST")) |
||||
|
def daypred(): |
||||
|
return predictday(request) |
||||
|
|
||||
|
@app.route("/deptpred", methods=("GET", "POST")) |
||||
|
def deptpred(): |
||||
|
return predictdept(request) |
||||
|
|
||||
|
@app.route("/titlepred", methods=("GET", "POST")) |
||||
|
def titlepred(): |
||||
|
return predicttitle(request) |
||||
|
|
||||
|
@app.route("/", methods=("GET", "POST")) |
||||
|
def index(): |
||||
|
return render_template("search.html") |
||||
|
|
||||
|
@app.route("/fc", methods=("GET", "POST")) |
||||
|
def fc(): |
||||
|
""" Filter Courses """ |
||||
|
print "trying to get courses" |
||||
|
params = dict(request.args.items()) |
||||
|
for key, val in params.iteritems(): |
||||
|
if val in defaults: |
||||
|
del params[key] |
||||
|
results = searchTerms(params) |
||||
|
return results |
||||
|
|
||||
|
@app.route("/resources", methods=("GET", "POST")) |
||||
|
def resources(): |
||||
|
""" Get Resources """ |
||||
|
notRequired = False |
||||
|
params = loads(dict(request.args.items())["data"]) |
||||
|
print params |
||||
|
author = params["author"] |
||||
|
title = params["title"] |
||||
|
|
||||
|
if ("No Textbooks" in title or |
||||
|
"No Adoption" in title): |
||||
|
return dumps("false") |
||||
|
|
||||
|
# Cache the result of the open library search |
||||
|
openlib = cacheit("openlib"+title+author, lambda : bookUrls(title, author)) |
||||
|
print openlib |
||||
|
|
||||
|
# cache the result of an internet archive search |
||||
|
iarchive = cacheit("iarchive"+title+author, lambda : searchIA(title, author)) |
||||
|
print iarchive |
||||
|
|
||||
|
if not (any(openlib) or any(iarchive)): |
||||
|
# We literally could not find ANYTHING |
||||
|
return dumps("false") |
||||
|
|
||||
|
return dumps({ |
||||
|
"iarchive" : iarchive, |
||||
|
"openlib" : openlib |
||||
|
}) |
||||
|
|
||||
|
@app.route("/scripts/<filename>") |
||||
|
def send_script(filename): |
||||
|
return send_from_directory(app.config["scripts"], filename) |
||||
|
|
||||
|
@app.route("/styles/<filename>") |
||||
|
def send_style(filename): |
||||
|
return send_from_directory(app.config["styles"], filename) |
||||
|
return app |
||||
|
|
||||
|
if __name__ == "__main__": |
||||
|
ClassSearch().run(port=8001, debug=True) |
||||
Loading…
Reference in new issue