Lead Generation with Web Scraping: Sources, Scoring and Contacts

A scraped lead list is only as good as its source and its filter. A directory gives you companies that describe themselves in a consistent format; a qualification step decides which of them fit what you sell; and only then is it worth paying to find the people to contact.

This guide maps lead sources to the sites we cover, each with its own step-by-step guide. Then it walks through one complete run, from 60 agencies on Clutch to a short list of decision makers, with the real numbers and costs from our own pipeline.

Every scraper linked here is one of our Actors on Apify. An Actor is a scraper that runs in Apify's cloud: you give it input such as a directory URL or company names, and it returns a dataset you can export as CSV, JSON or Excel. You pay per result, and each guide lists the current price.

Find the source for who you sell to

The best source is the one where your customers already describe themselves. These are the most common starting points; each guide shows a small run, the fields you get and what it costs.

You sell toStart with
Software, web or marketing agenciesClutch agency shortlist
Local businesses in one cityGoogle Maps business leads
US contractors and tradesBBB contractor shortlist
Businesses in Central Asia or the UAE2GIS listings with phones and emails
Companies that are hiringAll jobs at a company on LinkedIn
Named people at your target accountsDecision makers on LinkedIn
More lead sources

Agencies and freelancers: HubSpot Solutions Marketplace, Fiverr, Collabstr creators

Local businesses: Yelp, BBB details and complaints, Google Local Services Ads, property management companies, Booksy salons

Companies and signals: LinkedIn company data, Y Combinator companies, Amazon.de sellers with business details, LinkedIn job details, company career pages, AI lead enrichment and outreach drafts

What a directory record holds

Directories are good lead sources because every company describes itself in the same fields. Here is one company from our example run on October 7, 2026, scraped from the Chicago software developers page on Clutch with the Clutch.co Listings Scraper:

FieldValue
titleAtomic Object
locationChicago, IL
employeesCount50 - 249
hourlyRate$150 - $199 / hr
minProjectSize$25,000+
services50% custom software development, 50% mobile app development
rating / reviewCount4.9 / 52
websiteUrlTheir website, behind a Clutch click-tracking link

Team size, rates and minimum project size are exactly what most sellers use to qualify an agency, and they come in the same format for every company on the page. Note that Clutch lists an office location, which is not always where the founders are based: this agency's leadership is in Michigan.

Example: from 60 Clutch agencies to a short contact list

Our open-source Clutch lead generation pipeline runs the whole process locally, with Postgres in Docker. For the example, we pretended to sell white-label QA and test automation to software agencies, and described that ideal customer in a short Markdown file: custom software as the main service, 10 to 249 people, $50/hr or more, US-based, and no marketing agencies, staffing firms or 1,000+ person companies.

Lead report from the Clutch pipeline: agencies with fit scores, Claude's reason and outreach angle, and decision makers by title (names hidden)

1. Scrape the directory

Two Clutch directory pages, Chicago software developers and Austin web developers, returned 60 companies at $0.0015 each: $0.09.

2. Qualify before paying for contacts

Claude scored every company from 0 to 100 against the ideal customer profile and gave each a tier and a one-sentence reason. Of the 60, 18 landed in tier A, 27 in tier B and 15 were not a fit. The whole step cost $0.31 in Claude API fees.

CompanyScoreWhy (abridged from Claude's reason)
Atomic Object95, AUS-based, 50-249 people, all custom software and mobile development, $150-199/hr, $25,000+ projects
Eureka Software93, AAustin dev shop, 10-49 people, 90% custom software, web and mobile, $50,000+ projects
LaunchPad Lab72, BStrong rate and size, but development is only about 35% of services and the minimum project is $10,000
Techgropse55, BRight services and size, but the $25-49/hr rate leaves little margin for added QA
Coalition Technologies22, C70% SEO with only 10% web development: a marketing agency
Insight Global5, C1,000-9,999 people and primarily a staffing firm

The scoring followed the profile closely, including its exclusions: staffing firms, SEO agencies and large offshore shops all fell to tier C even when their size or location matched.

3. Find decision makers only at the best fits

Only the ten highest-scoring companies went to the LinkedIn Decision Maker Finder, at $0.025 per person found. We searched by each company's website rather than its name, with location set to United States: a name like "Synergy Labs" belongs to a dozen companies, while the website points to one LinkedIn page.

The search returned 17 people. Four were exact matches for the titles we asked for, a co-CEO, two CEO-founders and a co-founder, and came back with high confidence. The rest were medium confidence: mostly employees whose title did not match or was missing, plus at least one person who appeared to work for a different company with the same name. Small agencies often lack a clearly titled leader in search results, which is why the pipeline exports a confidence column: review medium rows by hand before any outreach.

4. Export to a CRM

The pipeline writes leads.csv with one row per contact: company, website, score, tier, Claude's reason and outreach angle, then the contact's name, title, LinkedIn URL and confidence. HubSpot, Pipedrive and most CRMs import it as is.

Pricing as of October 7, 2026: the full run cost about $0.09 for Clutch, $0.31 for Claude and $0.43 for the 17 decision makers. Running the contact search on all 60 companies instead of the top 10 would have cost roughly six times more for leads that were mostly not a fit.

Running it without code

  • Schedule the Actors in Apify. Directory scrapes can run weekly from the Apify Console and export to Google Sheets, a webhook or your CRM through Apify's integrations.

Collecting and using business contact data responsibly

These rules depend on where you and your contacts are, so treat this as a starting point rather than legal advice:

  • EU and UK. Business contact details of named people are personal data under the GDPR. Cold outreach usually relies on legitimate interests, which requires a balancing test, a clear way to object, and keeping only what you need. The UK ICO's guidance on legitimate interests is a practical place to start.
  • US email. The CAN-SPAM Act requires accurate headers and subject lines, a physical postal address and a working opt-out honored within 10 business days. See the FTC's compliance guide for business.
  • Records and sources. Keep where and when each record was collected, which makes deletion requests and stale-data cleanup straightforward.

Want this data without writing code?

Every scraper on this site runs on Apify with free starter credits.

Related guides