Web spiders — crawlers, bots, ants, scutters, whatever a given vendor calls them — are automated programs that extract information from web pages by crawling or scraping. In fraud and risk work, they show up on both sides of the fence.
Legitimate and fraudulent uses
On the legitimate side: search engines indexing content, site owners validating their own links and HTML, businesses tracking competitor pricing and product changes, researchers harvesting financial and investment data. On the fraudulent side: scammers harvesting email addresses for spam lists, and criminals copying legitimate site content wholesale to run pharming scams.
Merchant website monitoring
This combines automated and manual review to confirm a merchant is actually operating legally and complying with card association rules — checking for prohibited items like counterfeit goods, illegal drugs, or unlicensed gambling, and flagging merchants who misrepresent what they actually sell.
What compliance checks look at
- Alignment with Visa's Global Brand Protection Program
- MasterCard's Business Risk Assessment and Mitigation program requirements
- Regional legal compliance across the U.S., EU, and other jurisdictions
- Whether the merchant application matches what's actually being sold
- Pricing analysis to catch potential scam storefronts