Python机械化browser.forms（）

Question

I try to list all forms on page https://www.fmgroup.pl/login using Python mechanize module and browser.forms() function.

URL = 'https://www.fmgroup.pl/login'
br = mechanize.Browser()

br.open(URL)
for form in br.forms():
   print str(form)

I get this error:

Traceback (most recent call last):
  File "brows.py", line 25, in <module>
    for form in br.forms():
  File "build\bdist.win32\egg\mechanize\_mechanize.py", line 420, in forms
  File "build\bdist.win32\egg\mechanize\_html.py", line 557, in forms
  File "build\bdist.win32\egg\mechanize\_html.py", line 237, in forms
  File "build\bdist.win32\egg\mechanize\_form.py", line 844, in ParseResponseEx
  File "build\bdist.win32\egg\mechanize\_form.py", line 981, in _ParseFileEx
  File "build\bdist.win32\egg\mechanize\_form.py", line 758, in feed
  File "build\bdist.win32\egg\mechanize\_sgmllib_copy.py", line 110, in feed
  File "build\bdist.win32\egg\mechanize\_sgmllib_copy.py", line 144, in goahead
  File "build\bdist.win32\egg\mechanize\_sgmllib_copy.py", line 302, in parse_starttag
  File "build\bdist.win32\egg\mechanize\_sgmllib_copy.py", line 347, in finish_starttag
  File "build\bdist.win32\egg\mechanize\_sgmllib_copy.py", line 387, in handle_starttag
  File "build\bdist.win32\egg\mechanize\_form.py", line 735, in do_option
  File "build\bdist.win32\egg\mechanize\_form.py", line 480, in _start_option
mechanize._form.ParseError: OPTION outside of SELECT

I was trying to use different modules but they don't give me any forms in result.

Answer 1

I solwe this problem:

import requests
import sys
import urllib2
import re
import mechanize
import cookielib
from bs4 import BeautifulSoup


URL = 'https://www.fmgroup.pl/login'
address = 'https://www.fmgroup.pl/panel/developedtree'
br = mechanize.Browser()

response = br.open(URL)
soup = BeautifulSoup(response.read())
for div in soup.findAll('select'):
    div.extract()


response.set_data(str(soup))
br.set_response(response)

br.select_form(nr=0)

but i hava another problem with:

br.form['YumUserLogin[username]'] = LOGIN
br.form['YumUserLogin_password'] = PASSWORD

Error:

Traceback (most recent call last):
  File "brows.py", line 37, in <module>
    br.form['YumUserLogin[username]'] = LOGIN
  File "build\bdist.win32\egg\mechanize\_form.py", line 2780, in __setitem__
  File "build\bdist.win32\egg\mechanize\_form.py", line 3101, in find_control
  File "build\bdist.win32\egg\mechanize\_form.py", line 3185, in _find_control
mechanize._form.ControlNotFoundError: no control matching name 'YumUserLogin[username]'

from looks like:

<GET https://www.fmgroup.pl/search/search application/x-www-form-urlencoded
  <TextControl(q=)>
  <SubmitControl(yt0= ) (readonly)>>
<POST https://www.fmgroup.pl/user/auth/login application/x-www-form-urlencoded
  <HiddenControl(returnUrl=/panel) (readonly)>
  <HiddenControl(tag=ajax) (readonly)>
  <TextControl(YumUserLogin[username]=)>
  <PasswordControl(YumUserLogin[password]=)>
  <HiddenControl(YumUserLogin[rememberMe]=0) (readonly)>
  <CheckboxControl(YumUserLogin[rememberMe]=[1])>
  <SubmitControl(<None>=Zaloguj) (readonly)>>

Answer 2

I find out that this section of page mesing with this function. Somebody know how to skip this section?

<select id="selectcountry" name="selectcountry" style="width:200px;" onchange="MM_jumpMenu('parent',this,0)">
                    <option>Wybierz kraj:</option>
                    <option value="http://www.federicomahora.al" target="_blank">Albania</option>
                    <option value="http://www.fmgroup.com.ar" target="_blank">Argentyna</option>
                    <option value="http://www.fmgroupau.com.au" target="_blank">Australia</option>
                    <option value="http://www.fmgroupat.at" target="_blank">Austria</option>
                    <option value="http://www.fmgroup.be" target="_blank">Belgia</option>
                    <option value="http://www.fmbrazil.com.br" target="_blank">Brazylia</option>
                    <option value="http://www.fmgroup.bg" target="_blank">Bułgaria</option>
                    <option value="http://www.fmgroup.hr" target="_blank">Chorwacja</option>
                    <option value="http://www.fmgroup.gr" target="_blank">Cypr</option>
                    <option value="http://www.fmgroupmontenegro.com" target="_blank">Czarnogóra</option>
                    <option value="http://www.fmgroupcz.com" target="_blank">Czechy</option>
                    <option value="http://www.fmgroup.ee" target="_blank">Estonia</option>
                    <option value="http://www.fmgroupfragrance.ph" target="_blank">Filipiny</option>
                    <option value="http://www.fmgroup.fr" target="_blank">Francja</option>
                    <option value="http://www.fmgroup.gr" target="_blank">Grecja</option>
                    <option value="http://www.fmgeorgia.com" target="_blank">Gruzja</option>
                    <option value="http://www.fmgroup.es" target="_blank">Hiszpania</option>
                    <option value="http://www.fmgroup.nl" target="_blank">Holandia</option>
                    <option value="http://www.fmgroup.co.id" target="_blank">Indonezja</option>
                    <option value="http://www.fmiceland.is" target="_blank">Islandia</option>
                    <option value="http://www.fmgroupcanada.com" target="_blank">Kanada</option>
                    <option value="http://www.fmkenya.com" target="_blank">Kenia</option>
                    <!-- <option value="http://fmgroup.cr" target="_blank">Kostaryka</option> -->
                    <option value="http://www.fmgroup.lt" target="_blank">Litwa</option>
                    <option value="http://www.fmcosmetics.my" target="_blank">Malezja</option>
                    <option value="http://www.maroc.fmworld.com" target="_blank">Maroko</option>
                    <!--<option value="http://www.fmgroup.md/" target="_blank">Mołdawia</option>-->
                    <option value="http://www.fmgroup.mn" target="_blank">Mongolia</option>
                    <option value="http://www.fmgermany.de" target="_blank">Niemcy</option>
                    <option value="http://www.fmgroupng.com" target="_blank">Nigeria</option>
                    <option value="http://www.fmgroup.pl" target="_blank">Polska</option>
                    <option value="http://www.federicomahorar.ru" target="_blank">Rosja</option>
                    <option value="http://www.fmgroupcosmetics.ro" target="_blank">Rumunia</option>
                    <option value="http://www.fmgroupsk.com " target="_blank">Słowacja</option>
                    <option value="http://www.fmgroup.si" target="_blank">Słowenia</option>
                    <option value="http://www.fmgroupswitzerland.ch" target="_blank">Szwajcaria</option>
                    <option value="http://www.fmgroupthailand.com" target="_blank">Tajlandia</option>
                    <option value="http://www.fmgroup.lviv.ua" target="_blank">Ukraina</option>
                    <option value="http://www.fmgrouphungary.com" target="_blank">Węgry</option>
                    <option value="http://www.fmcosmetics.co.uk" target="_blank">Wielka Brytania</option>
                    <option value="http://www.fmgroup.co.it" target="_blank">Włochy</option>
                    <option value="http://www.fmgroupuae.com" target="_blank">Zjednoczone Emiraty Arabskie</option>
        </select>