Almost any Osint (Open-Source) research or Webint (Web Intelligence) investigation starts with a search engine, usually Google, which is indeed the best thing to start with. In this guide, we will describe some of the recommended methods and advanced search engines for skilled online research.
Search Operators
All major search engines enable the use of operators – special characters and commands for a more targeted search. The use of these advanced operators is sometimes crucial to valuable research.
Google Operators
Google has dozens of operators that can be used in different use cases, for market or company research. The most usable operators in our opinion are:
- Site: – Limit results to those from a specific domain (site:microsoft.com ‘AR headset’).
- Great for searching for activities or people related to a company.
- Can be used also for searching in specific platforms and social networks, which often have mediocre internal searching options.
- Filetype: / ext: – Only search for specific filetype (filetype:pdf “one page”)
- Great for finding official documents, presentations, or sheets.
- Can be used also for threat intelligence or for locating more juicy or sensitive info, for example – ext:psw | ext:sql | ext:dbf | ext:bkp.
- Cache: – Returns the most recent cached version of a web page – a snapshot of the page as it appeared the last time it was indexed (cache:tesla.com)
- You can then show the webpage as ‘Text Only’ or to view its source and look for any changes that happened to the site.
- Related: – Find sites related to a given domain (related:salesforce.com)
- Useful for finding similar sites or competitors
- Inurl: – Find pages with a term in the URL (inurl:blog ‘smart cars’)
- Helpful for searching for specific types of results – blogs, online shops, about sections, speakers, etc.)
Full list of Google operators with examples can be found here.
Like Google, Bing, Yandex, DuckDuckGo, and Baidu also have their own operators. Unfortunately, there are far fewer operators than Google. Check here on page 3 to explore their operators.
You can also combine a few operators for more specific searches. For example – “site:microsoft.com ext:ppt ‘azure’” to get presentations about ‘azure’ from Microsoft’s website.
Advanced Search Engines
Classic search engines are not the only ones. In recent years, we’ve seen multiple advanced search engines, some harnessing the power of machine learning to perform a much smarter search – clustering results into topics, advanced filtering, and unique data sources.
Carrot2 – Organizes your search results into topics.
Carrot2 uses algorithms to organize search results into thematic folders. With an instant overview of what’s available, you will quickly find what you’re looking for. You can display the results as Treemap or Pie-chart as well.
Yippy – Concept Clustering
Powered by IBM Watson, Yippy allowing users to navigate big data sets and search results by topical categories dynamically. You can also filter by type of site, dates, and more.
DeepWebTech – Explorit Everywhere!
Simultaneous searching of all sources, Intelligent Clustering of search results, Customizable Search Builder for multiple user groups, Automatic Alerts and Incremental Search, Usage statistics and Administration tools, Highly flexible sort and filter options.
DeepWebTech has built several showcase federated search applications – Business Search, Medical Search, Science Information Search, and more.
Million Short – A better search experience.
Search engine with an interesting concept-allows you to remove 100, 1000, 10 K, 100 K, or 1 million first results from Google. The idea behind it is that most people see more or less the same content, from big sites only, and to find the more interesting content you need to get to the less common and familiar areas of the web. Million Short lets you customize the way you search the web. Results can be filtered by popularity, shopping, advertising, privacy, country, and communication.
2lingual – Makes it easy to Google Search in 2 languages
Get Google Search Results alongside Google Cross-Language Search Results. Also, a Query Translation Option can be activated or deactivated for Google Cross-Language Searches. Currently, 37 Google Search Languages are supported.
More Search Recommendations for Osint and Webint
Greedy Search – On the first iteration of your search, you might want to do a ‘greedy search’ and look for the specific thing you want to know (even in quotes), for example – ‘Microsoft’s cloud services top competitors in Europe’. There’s a chance that you’ll find an article, blog, or other discussions exactly on this subject, or on a related topic. On the next search, you can search for broader terms and use keywords instead of the full sentence.
Adjust your search based on results – you searched for a Company / Person and found an interesting relation, new business activity, additional personal information – try to search again with new keywords you’ve found. Don’t be static, adjust the search for any information you find.
Don’t settle for the first pages of the results – the first results you get in search engines are usually less interesting. Continue researching the results on the next pages.
Combine tools – Have you found some interesting information? Keep exploring this in other, more dedicated tools for this subject.
In conclusion
The conduct of a qualified search and the disclosure of significant information takes time and requires some experience. Using the tools and methods mentioned above can make our search more in-depth and comprehensive. You’ll be surprised how much you can find by searching and exploring with the right tools, never give up before you make sure there’s nothing left to try.
