Web Scraping / JavaScript Rendering
How to Scrape JavaScript-Rendered Pages
Most modern websites rely on JavaScript to load their content. If you send a plain HTTP request to YouTube, Amazon, or LinkedIn, the response is a near-empty HTML shell - the actual data gets loaded by JavaScript after the page opens in a browser. Here is how to get the fully rendered page without setting up your own browser infrastructure.
Why plain HTTP requests are not enough
JavaScript builds the page after it loads
When you use Python's requests library, Node.js fetch, or PHP's file_get_contents, you get the raw HTML the server sends before any JavaScript runs. For a site like YouTube, that raw HTML contains almost nothing - the video titles, channel info, and comments are all injected by JavaScript afterward.
Running your own browser is complex
The traditional solution is to run a headless browser like Puppeteer, Playwright, or Selenium. But that means installing Chrome or Chromium, keeping browser versions in sync with drivers, managing memory leaks and zombie processes, and handling crashes under load. It works, but it is a significant amount of infrastructure to maintain.
Headless browsers get detected
Even with a working headless browser, many websites detect and block automated access. Default Puppeteer and Playwright installations expose dozens of fingerprinting signals - from the navigator.webdriver flag to canvas rendering differences - that anti-bot systems use to distinguish real browsers from automated ones.
Get fully rendered HTML in one API call
Browser7 runs a real Chrome browser for every request. JavaScript is executed, AJAX calls complete, and dynamic content loads - all before the HTML is returned to you. There is no browser to install, no driver to update, and no fingerprint leaks to worry about.
from browser7 import Browser7
client = Browser7(api_key="b7_your_api_key")
result = client.render("https://www.youtube.com/@MrBeast")
# Full rendered HTML - all JavaScript executed, all content loaded
print(len(result.html), "characters")The response contains the same HTML you would see if you opened the page in your own browser and ran document.documentElement.outerHTML in the console. Every element rendered by JavaScript is included.
Wait for specific content to load
Some pages load content in stages - the initial JavaScript runs, then additional data arrives via AJAX calls. If the data you need appears after a delay, use the wait_for_selector option to tell Browser7 to wait until a specific element appears in the page before returning the HTML.
from browser7 import Browser7
client = Browser7(api_key="b7_your_api_key")
# Wait for job cards to load before returning the HTML
result = client.render(
"https://www.indeed.com/jobs?q=software+engineer&l=Remote",
wait_for_selector="#mosaic-provider-jobcards",
country_code="US",
)
print(result.html)In this example, Indeed loads its job listings dynamically after the initial page render. Without the wait, you would get the page skeleton but no job data. With the wait, Browser7 holds until the job cards container appears, then returns the complete HTML with all listings included.
Sites that need JavaScript rendering
If you are unsure whether a site needs JavaScript rendering, try viewing its source code in your browser (right-click, View Page Source). If the data you want is not in the raw HTML, the site uses JavaScript to load it.
Single-page applications
React, Vue, Angular, and other SPA frameworks render everything client-side. The server sends a minimal HTML shell and a JavaScript bundle that builds the page in the browser.
Infinite scroll and lazy loading
Sites like YouTube, Twitter/X, and TikTok load content as you scroll. The initial HTML contains only the first batch of items - the rest loads dynamically.
Dynamic pricing and availability
E-commerce sites and travel platforms often fetch pricing data via JavaScript after the page loads, sometimes personalized based on cookies or location.
Client-side search results
Job boards, real estate sites, and marketplaces frequently render search results with JavaScript, pulling data from internal APIs after the initial page load.
What this costs
JavaScript rendering is included in every Browser7 request at no extra charge. Every page costs $0.01 - whether the site is a static HTML page or a JavaScript-heavy single-page application. There are no rendering surcharges, no credit multipliers, and no feature tiers.
Many scraping APIs charge 5-25x more for JavaScript rendering because they treat it as a premium feature. Browser7 includes it by default because in 2026, most websites need it.
See it in practice
These guides show JavaScript rendering in action on real websites, with complete parsing code:
- How to Scrape YouTube - JavaScript SPA with dynamic video listings
- How to Scrape Indeed - Job cards loaded via JavaScript after page render
- How to Scrape TikTok - Fully client-rendered profile pages