Critical URLs exposed by a misconfiguration in Urlscan.io

November 17, 2022
Critical URLs Exposed Data Misconfiguration Urlscan.io Data Leak

The website scan and analysis engine software, Urlscan.io has accidentally leaked sensitive records of scanned URLs after a misconfiguration in its system. Urlscan.io caters to URL submissions and produces troves of data such as domains, DOM information, cookies, screenshots, and IPs.

According to its developers, the software aims to enable a user to analyse unidentified and potentially hostile websites quickly. This website aids many enterprise customers and open-source projects. Additionally, this system provides an API to integrate checks into third-party products.

 

Leaked GitHub pages were the first indication of compromise in the Urlscan.io software.

 

Based on reports, an email sent by GitHub earlier this year warned about its uses regarding GitHub pages that had been accidentally leaked through a third-party (Urlscan.io) during a metadata analysis. The leak might have included password reset links, setup pages, meeting invitations, urlscan.io dorks, DocuSign signing requests, PayPal invoices, and Telegram bots.

Experts confirmed that the scans to the affected software were the cause of the leaked data after pingbacks to exposed email addresses revealed that misconfigured tools were in the scans of the software.

The proof of this accusation is that many affected API integrations used the standard python-requests/2.X[.]Y user agents, enabling scans to be unintentionally posted as public.

A research team contacted some of the leaked emails, and only one has responded. The responder’s employer discovered that a misconfiguration of the Security Orchestration, Automation, and Response (SOAR) playbook integrated with Urlscan was faulty. This incident led to further investigation and examination of Urlscan.io’s history to uncover misconfigured clients that anyone could exploit by scraping the system for email addresses.

The scary part of this issue is that users with misconfigured clients could experience password resets for many web services that can be activated. Additionally, the exposed link could be utilised by an individual to set a new password and control the accounts.

Fortunately, the impact assessment was finished by the company’s team a couple of months ago. Researchers and the Urlscan.io developers are working to address the identified misconfigurations.

As of now, a new version of the software might appear in the coming months.

About the author

Leave a Reply