How to look up information from a website with python

Question

I want to know if its possible to write a code in python that will allow me to look up information from an online source and add it too my code as a dictionary. (I want to use this so I have a dictionary consisting of all the spells listed on the harry potter wiki as the key and their descriptions as associated values ( https://harrypotter.fandom.com/wiki/List_of_spells ))

I am beginning python student and really don't know how to start, I guess I could copy the information as a text file and manipulate it from there but I really want it too change should the online source change etc.

Answer 1

You can grab the wiki data here and parse it:

https://harrypotter.fandom.com/wiki/List_of_spells?action=edit

It looks like the spells follow the same format making it easy for parsing. They are separated by a new line so you can split the data by \n, and parse each line out. There seems to be two different type of spells, ones that start with '|'and others that start with ':', so you have to parse differently for the type. Does that help you get started?

===''[[Water-Making Spell|Aguamenti]]'' (Water-Making Spell)===
[[File:Aguamenti.gif|235px|thumb]]
{{spell sum
|t=Charm, Conjuration
|p=AH-gwah-MEN-tee
|d=Produces a clean, drinkable jet of water from the wand tip.
|sm=Used by [[Fleur Delacour]] in [[1994]] to extinguish her skirt, which had caught flame during a fight against a [[dragon]]. [[Harry Potter|Harry]] used this spell twice in [[1997]], both on the same night; once to attempt to provide a drink for [[Albus Dumbledore|Dumbledore]], then again to help douse [[Rubeus Hagrid|Hagrid]]'s hut after it was set aflame by [[Thorfinn Rowle]], who used the [[Fire-Making Spell]].
|e=Possibly a hybrid of [[Latin]] words ''aqua'', which means "water", and ''mentis'', which means "mind".}}

===''[[Alarte Ascendare]]''===
[[File:Alarte Ascendare.gif|250px|thumb]]
{{spell sum
|t=Charm
|p=a-LAR-tay a-SEN-der-ay
|d=Shoots the target high into the air.
|sm=Used by [[Gilderoy Lockhart]] in [[Harry Potter and the Chamber of Secrets (film)|1992]]
|e=''Ascendere'' is a [[Latin]] infinitive meaning "to go up,""to climb," "to embark," "to rise(figuratively);" this is the origin of the English word "ascend".}}

===([[Albus Dumbledore's forceful spell|Albus Dumbledore's Forceful Spell]])===
:'''Type:''' Spell
:'''Description:''' This spell was, supposedly, quite powerful as when it was cast, [[Tom Riddle|the opponent]] was forced to conjure a [[Silver shield spell|silver shield]] to deflect it.
:'''Seen/Mentioned:''' This incantation was used only once throughout the series, and that was by Dumbledore in the [[British Ministry of Magic|Ministry of Magic]], immediately following the [[Battle of the Department of Mysteries]] on [[17 June]], [[1996]], while he duelled Voldemort.

===''[[Unlocking Charm|Alohomora]]'' (Unlocking Charm)===
[[File:Unlocking charm1.gif|235px|thumb]]
:'''Type:''' Charm
:'''Pronunciation:''' ah-LOH-ho-MOR-ah
:'''Description:''' Unlocks doors and other objects. It can also unlock doors that have been sealed with a [[Locking Spell]], although it is possible to bewitch doors to become unaffected by this spell.
:'''Seen/Mentioned:''' Used by [[Hermione Granger]] in [[1991]] to allow [[Trio|her and her friends]] to access the [[Third-floor corridor]]] at [[Hogwarts School of Witchcraft and Wizardry|her school]], which was at the time forbidden; she used it again two years later to free [[Sirius Black|Sirius]]'s cell in [[Filius Flitwick's office|her teacher's prison room]].
:'''Etymology:''' The incantation is derived from the West African Sidiki dialect used in geomancy; it means "friendly to thieves", as stated by [[J. K. Rowling|the author]] in testimony during a court case.<ref name=alomohoracourt>{{cite web |url=http://cyberlaw.stanford.edu/files/blogs/Trial%20Transcript%20Day%201.txt |title=Warner Bros Entertainment, Inc. and J.K. Rowling  v. RDR Books (Transcript) |author=United States District Court, Southern District of New York |date=April 14, 2008 |publisher=Stanford Law School |access-date=October 1, 2015 |quote=Alohomora is a Sidiki word from West Africa, and it is a term used in geomancy.  It is a figure -- the figure alohomora means in Sidiki "favorable to thieves."  Which is obviously a very appropriate meaning for a spell that enables you to unlock a locked door by magic.}}</ref>
:'''Notes:''' Whilst in the first book, when the spell is cast the lock or door must be tapped once, in the fifth, [[Miriam Strout|a healer]] simply points her wand at the door to cast it, and on {{PM}} the wand motion is seen as a backward 'S'.