python - Errors with Beautiful Soup output -

-

i'm trying scrape data webpage on gamespot using beautifulsoup. however, result different page source viewer. first off, alot of errors produced. instance, have

r = requests.get(link)   soup = bs4.beautifulsoup(r.text)

and yet soup.title gives

<title>404: not found - gamespot</title>.

the data want scrape not appear. because webpage contains javascript alongside ? if how can around ?

you're sending http request server. need process javascript content.

a headless browser javascript support, ghost, it'd choice.

from ghost import ghost  ghost = ghost()  ghost.open(link) page, resources = ghost.evaluate('document.documentelement.innerhtml;') soup = beautifulsoup(page)

.evaluate('document.documentelement.innerhtml') show dynamically generated content, not static you'd see taking @ source.


Comments

Post a Comment

Popular posts from this blog

android - Gradle sync Error:Configuration with name 'default' not found -

-
i want add external library guillotinemenu-android in application. followed steps given in upvoted answer existing question how add library project android studio? facing error when try build project after step 6 i.e. when added dependency in app/build.gradle compile project(":guillotinemenu") . tried stackoverflow links related error not working out. have made folder named libs in app directory , copied project folder guillotine menu there. here build.gradle(module:app) file apply plugin: 'com.android.application' android { compilesdkversion 22 buildtoolsversion "22.0.1" defaultconfig { applicationid "com.sunshine.bbreaker.appet_i" minsdkversion 14 targetsdkversion 22 versioncode 1 versionname "1.0" } buildtypes { release { minifyenabled false proguardfiles getdefaultproguardfile('proguard-android.txt'), 'proguard-rules.p...
Read more

StringGrid issue in Delphi XE8 firemonkey mobile app -

-
i'm using delphi xe8 developing mobile application , i'm facing of problem in tstringgrid. i have written following code in stringgridselectcell event. showmessage(stringgrid.cells[0, arow]); and shows proper value of selected row @ first time. when have tried click again selected row, event not getting fired. , cant able unselect particular row. so, have tried write message in onclick event. var irowind: integer; begin irowind := stringgrid1.selected; showmessage(stringgrid.cells[0, irowind]); end; and not working @ first time of click , when clicked second time works properly. issue in android mobile , not in windows. later, have kept both event code , faced issue. when i'm scrolling grid, firing onclick event. so, displaying selected row value. please provide me solution. , in advance. let's onclick event: procedure tfmmain.stringgrid1click(sender: tobject); begin caption := 'selected row id: ' + stringgrid1.select...
Read more

html - jQuery UI Sortable - Remove placeholder after item is dropped -

-
i'm trying nested sortable , i'm kind'a succeeding, there's 1 small little inconvenience bugs me. i want placeholder disappear after i've dropped dragged item (on mouseup) , can't quite figure out how it. i want because when sort downwards, deletion of placeholder affects parent's height, in turn creates small bug, check jsfiddle here. html <div class="container"> <h1>menu</h1> <ul class="list-group striped nest"> <li class="list-group-item">home <ul class="list-group striped nest"></ul></li> <li class="list-group-item">about <ul class="list-group striped nest"></ul></li> <li class="list-group-item"> services <ul class="list-group striped nest"> <li class="list-group-item">design <ul class=...
Read more