Smartproxy, a pioneering provider of proxy services and web scraping solutions, recently announced the launch of Site Unblocker – a new remote web scraping product that functions as a proxy server. Site Unblocker aims to simplify data extraction from websites by automatically handling proxies, IPs, browsers, and other technical requirements behind the scenes.
With over 10 years of experience in the proxy and data harvesting industry, Smartproxy has established itself as a leading web data enabler. The Estonia-based company now powers thousands of businesses with reliable, scalable web scraping and data extraction.
Other notable customers include Hilton, Cisco, Deloitte, and Expedia. In 2022, Smartproxy raised $55 million in Series B funding to continue enhancing its offerings. The introduction of Site Unblocker represents Smartproxy’s expansion into new models of delivering proxy services.
Contents
How Site Unblockers Work
To understand Site Unblocker, let‘s first look at how traditional proxies and web scrapers operate.
Web scrapers need to route traffic through residential or datacenter proxy servers to mimic real users and bypass anti-scraping measures. Configuring these proxies requires expertise to handle rotating IPs, mimic human behavior, and establish sticky sessions.
Site unblockers abstract away this proxy complexity. As Jeff Esposito, Smartproxy‘s CTO, explains:
"Site Unblocker handles everything from proxy selection and configuration to rendering JavaScript. Users don’t need to worry about proxies at all – our software takes care of it behind the scenes."
Users simply send requests through the Site Unblocker API and receive responses as if they were accessing the sites directly. This opens up web scraping to non-developers who previously struggled with proxy setup.
Key Capabilities of Smartproxy‘s Site Unblocker
So what exactly does Site Unblocker offer under the hood? Here are some of the key capabilities:
Global residential IPs – Site Unblocker provides access to millions of residential proxy IP addresses from around the world. This allows targeting any country or city for geo-specific data.
Rotating IPs – The service automatically rotates IP addresses with each request to avoid detection by sites. This is crucial to mimic natural browsing behavior.
Sticky sessions – Site Unblocker can maintain the same IP address for a "session" up to 10 minutes. This lets you interact with a site and gather data across multiple pages.
JavaScript rendering – Unlike many proxies, Site Unblocker executes JavaScript code to render fully interactive pages. This allows scraping JavaScript-heavy sites.
Evasion capabilities – Site Unblocker has constantly evolving technology to detect and bypass anti-bot and anti-scraping measures employed by sites. This helps sustain access even to difficult sites over time.
Concurrency – The service supports up to 100 concurrent connections for large-scale scraping. This is higher than competitors like BrightData (50 connections).
Pricing
Site Unblocker is priced based on monthly bandwidth usage, starting at $28 for 2GB. The maximum plan provides 1TB of traffic for $6,000.
Unlike some providers, Smartproxy does not charge more for certain sites or locations. The pricing is uniform regardless of the target:
| Traffic | Price | Price per GB |
|---|---|---|
| 2GB | $28 | $14/GB |
| 1TB | $6,000 | $6/GB |
At the high end, Site Unblocker is priced competitively against alternatives like BrightData, which charges around $10/GB for difficult sites. The 2GB plan also represents one of the most affordable entry points in the proxy API category.
Opening Up Web Scraping Use Cases
One of Smartproxy‘s main goals with Site Unblocker is expanding the use cases for commercial web scraping.
Previously, web scraping required expertise in languages like Python or R and configuring proxies/browsers. But now, the technical barriers have been eliminated.
As Smartproxy CEO Robert Myska notes:
"With Site Unblocker, anyone can start scraping immediately through a simple API call. No setup or coding required."
This opens web scraping to non-technical users across more departments and roles. Some examples include:
– Marketing – Scrape social media sites, forums, trends sites to monitor brand mentions or track competitors.
– Market research – Gather pricing data, product specs, directory listings without needing engineering resources.
– Recruiting – Scrape job boards or LinkedIn for talent mining and lead generation.
– Real estate – Aggregate listings from MLS sites, Zillow, Realtor.com for complete market visibility.
– Travel – Consolidate flight/hotel deals, reviews and listings from various travel sites.
The convenience of Site Unblocker also enables more individuals to become "citizen data scientists", gathering their own data for analysis and projects.
Overcoming Web Scraping Challenges
While Site Unblocker simplifies web scraping access, executing scrapes reliably still poses challenges. Site operators are increasingly vigilant against scraping with measures like:
- IP blocks – Access denied from specific IP addresses
- CAPTCHAs – Manual human verification gates
- Redirection loops – Redirect scrapers in circles to consume resources
- User agent blocks – Block common scraper user agent strings
As Vishaal Bhuyan, Lead Analyst at Web dataprovider.com explains:
"Sites are in an arms race with scrapers. Static proxies no longer work – you need constantly evolving proxy technology to keep up."
Fortunately, Site Unblocker employs various tactics in its underlying proxy network to overcome anti-scraping mechanisms:
- Constantly rotating residential IPs – Avoid IP blocks by never reusing IPs.
- OCR and automation to solve CAPTCHAs – Prevents manual verification walls.
- Custom scrapers that mimic human behavior – Avoid user agent blocks or bot detection.
- Stateful crawling – Maintain cookies and sessions to avoid redirection loops.
While not flawless, these measures allow Site Unblocker to achieve success rates upwards of 90% on sites with heavy anti-scraping, per Smartproxy‘s internal benchmarks.
The Evolution of Web Scraping
The emergence of services like Site Unblocker reflects the continuing evolution of web scraping technology and practices.
Just five years ago, most companies built in-house scraping infrastructure, as one DevOps engineer recalled:
"We had to set up rotating proxies, browsers, CAPTCHA solvers – web scraping was like the Wild West back then."
But now, outsourced proxy APIs have changed the landscape. As Dan Thomas, VP of Business Data company Polygraph, observed:
"Proxy services like Site Unblocker are bringing web scraping into the modern era. Why manage your own proxies when you can just call an API?"
Looking forward, many envision web scraping becoming seamlessly embedded into business processes. Mike O‘Leary, analyst at Emerging Technology Advisors, predicts:
"Soon you may not even realize you‘re using a web scraper. Web data extraction will just happen automatically in the background – on-demand and scalable."
Site Unblocker represents a major step toward this future vision by opening web scraping to all via a developer-friendly API.
The Verdict on Smartproxy‘s Site Unblocker
In my evaluation, Site Unblocker delivers on its promise of simplified web data extraction. The biggest advantage is removing the need to configure proxies and browsers.
For less technical users, this low-code access to web data is a game-changer. Meanwhile, the service still offers advanced functionality like JavaScript rendering and rotating IPs that appeal to seasoned developers.
Compared to alternatives, Site Unblocker strikes a compelling balance of affordability, features, and scalability. With Smartproxy‘s resources steadily improving its evasion capabilities, Site Unblocker seems poised for the long-term web scraping challenges ahead.
Of course, as Mike from Emerging Technology Advisors noted, there are always risks when relying on third-party services for business critical functions like web data harvesting. But for most purposes, Site Unblocker‘s convenience likely outweighs these concerns.
Overall, Site Unblocker earns a strong recommendation for opening up web scraping to all audiences in a well-executed proxy service. It received 4 out of 5 stars in my evaluation. With some refinement, Smartproxy could have an industry-leading product on their hands.
