Thinking Outside the Box with Public Data

Demonstrating the diverse applications of UK public data, from individual cases to national-scale insights.

Category
Guides & Information
Date
October 18, 2024
Author
Sham Ahmed

Author: Sham Ahmed (LinkedIn)

Whether it’s providing situational awareness in fast-moving environments, uncovering hidden connections in complex investigations, or monitoring real-time data streams, OSINT usually draws from a wide range of sources to build a comprehensive intelligence picture. However, many OSINT practitioners tend to stick with the data sources they know best and that have historically provided relevant results, creating a reliance on familiar tools and sources. This ‘source bias’ can limit the potential for new insights from underutilised datasets.

This means that even for seasoned investigators, certain publicly available datasets, particularly government-held records, are often underutilised. These datasets contain a wealth of data that can be harnessed for investigations into everything from financial misconduct to environmental regulation compliance. By thinking creatively and going beyond the most commonly used OSINT sources, investigators and analysts can unlock the full potential of public data. In this article, I’ll explore how thinking creatively with public data can unlock broader insights.

Exploring The Value of Government Data

The UK is a treasure trove of public data, much of which is available through local councils, government departments, and public registers. While some of these sources may seem mundane at first glance, they can reveal important patterns and insights when viewed at scale or combined with other datasets.

From planning registers to insolvency records and licensed premises data, OSINT investigators can gather information that helps them answer key questions about individuals, businesses, or environmental changes. By examining the data creatively, they can uncover hidden patterns, potential violations of law, and critical leads for their investigations.

One of the challenges for OSINT investigators is that much of the most useful data is decentralised and difficult to search for. Much of the publicly available data from the UK government is available locally, via council portals, or via the ONS. To gather insights on a national scale, investigators would traditionally have to visit each council’s portal separately and work within the allowed search parameters.

Tree Felling Case Study

To explore an example of the potential of this disparate data, the Public Insights team compiled data on planning permission for tree felling from across the 380 different public council planning registers, creating a map that provides a national view of permitted tree fellings. This map allows investigators to see trends and insights that would be impossible to glean from a single local council's records.

A map showing trees that have been granted planning permission to be cut down might not seem like a typical source for an OSINT investigation. When viewed at a national level, patterns and trends that might otherwise be invisible at the local level become clear, providing new investigative angles for those focused on environmental regulation, land use, or illegal activity.

Here’s how different sectors can benefit from this insight:

  • Environmental charities: Reviewing tree removals nationally can help identify areas at risk of deforestation or environmental degradation. By having a national view, charities can identify areas at risk to target and advocate for stronger environmental protections to protect habitats or advocate for stricter environmental regulations.
  • Land developers and property analysts: Tracking areas of significant tree felling might indicate new land development projects or urban expansion. This could provide useful insights for assessing the impact on local property markets and potential land value changes.
  • Investigators of planning or preservation order violations: By cross-referencing tree felling with planning permissions, preservation orders, and conservation zones, investigators can spot potential illegal activity, helping authorities hold violators accountable.

With Cradle, we’ve found that combining these types of public datasets, from planning registers to housing records, creates a much more complete investigative picture. This approach to gathering and analysing decentralised data helps connect disparate information into a coherent narrative.

Broader Applications of Public Data

This approach of gathering localised public data and turning it into a broader resource can be applied across many different types of government records. By thinking outside the box and using public data creatively, OSINT practitioners can find key insights for investigations. Let’s look at some additional public datasets and how they can be used:

  • Land Registry: Land registry records show ownership of properties and land, as well as transaction history. For OSINT investigators, these property ownership records are invaluable for connecting individuals or businesses to properties, helping to trace assets, uncover hidden ownership, or investigate real estate fraud.
  • Insolvency Data: Insolvency records reveal individuals and businesses that have been declared bankrupt or have entered into liquidation. This can be crucial for investigations into financial misconduct or when assessing a business's credibility. Insolvency data also provides clues when investigating fraudulent business practices, helping to flag companies with a history of repeated failures or misconduct.
  • Licensed Premises: Records of licensed premises, such as pubs and restaurants, can be useful for investigators looking into individuals or businesses involved in the hospitality industry. These records provide evidence of business activity and ownership, helping investigators understand revenue streams, identify potential money laundering operations, or ensure compliance with licensing regulations.
  • Licensed Taxis: Taxi license records can be used to determine the business activity of individuals operating in the taxi industry, helping investigators link individuals to specific vehicles or regions. These records help investigators verify claims of business operation and identify any unlicensed or fraudulent activity.
  • HMO Registers: Houses in Multiple Occupation (HMO) registers list properties that are rented to multiple tenants. These registers are useful for identifying landlords, estimating undeclared income, and investigating non-compliance with local housing regulations.

Thinking Outside the Box with Public Data

The power of OSINT lies in its ability to piece together seemingly disparate datasets to reveal a clearer picture of individuals, businesses, or activities. Public data, particularly from government databases, provides a rich, unique, and often underutilised source of information for investigators. 

By compiling data from these sources, as with the tree felling map, investigators can uncover patterns and insights that may not be immediately obvious. For example, combining land registry data with licensed premises and HMO information could help identify a business owner who owns both rental properties and commercial establishments, revealing potential undeclared income streams. 

Government datasets in the UK offer an abundance of valuable information for OSINT investigators, from property ownership and business licensing to environmental changes and financial records. The key is knowing where to look and how to connect the dots. By thinking creatively and processing data at scale, OSINT practitioners can unlock new insights and develop a comprehensive view of the subjects they’re investigating.

"
An outstanding atmosphere and an amazing gathering of professionals
Daniel Heinen
Founder, GeoSpy
Bridging industry
gaps to create a real community
Stephen Adams
Founder, Intelligence With Steve
Synergy between different minds and tradecraft
William R
Founder, The Aracari Project
The first meet-up was informative, engaging, and community-driven.
Peter Allwright
Head of Suntera Forensics
Prev
Next