What is the tech stack behind Google Search Engine?

Google Search Engine Tech Stack

The original Google algorithm was called PageRank, named after inventor Larry Page (though, fittingly, the algorithm does rank web pages). 

r/dataisbeautiful - [OC] Google dominates the search market with a 91.9% market share

After 17 years of work by many software engineers, researchers, and statisticians, Google search uses algorithms upon algorithms upon algorithms.

2022 AWS Cloud Practitioner Exam Preparation

How does Google’s indexing algorithm (so it can do things like fuzzy string matching) technically structure its index?

  • There is no single technique that works.
  • At a basic level, all search engines have something like an inverted index, so you can look up words and associated documents. There may also be a forward index.
  • One way of constructing such an index is by stemming words. Stemming is done with an algorithm than boils down words to their basic root. The most famous stemming algorithm is the Porter stemmer.
  • However, there are other approaches. One is to build n-grams, sequences of n letters, so that you can do partial matching. You often would choose multiple n’s, and thus have multiple indexes, since some n-letter combinations are common (e.g., “th”) for small n’s, but larger values of n undermine the intent.
  •  don’t know that we can say “nothing absolute is known”. Look at misspellings. Google can resolve a lot of them. This isn’t surprising; we’ve had spellcheckers for at least 40 years. However, the less common a misspelling, the harder it is for Google to catch.
  • One cool thing about Google is that they have been studying and collecting data on searches for more than 20 years. I don’t mean that they have been studying searching or search engines (although they have been), but that they have been studying how people search. They process several billion search queries each day. They have developed models of what people really want, which often isn’t what they say they want. That’s why they track every click you make on search results… well, that and the fact that they want to build effective models for ad placement.
  • Each year, Google changes its search algorithm around 500–600 times. While most of these changes are minor, Google occasionally rolls out a “major” algorithmic update (such as Google Panda and Google Penguin) that affects search results in significant ways.

    For search marketers, knowing the dates of these Google updates can help explain changes in rankings and organic website traffic and ultimately improve search engine optimization. Below, we’ve listed the major algorithmic changes that have had the biggest impact on search.

  • Originally, Google’s indexing algorithm was fairly simple.

    It took a starting page and added all the unique (if the word occurred more than once on the page, it was only counted once) words on the page to the index or incremented the index count if it was already in the index.

    The page was indexed by the number of references the algorithm found to the specific page. So each time the system found a link to the page on a newly discovered page, the page count was incremented.

    When you did a search, the system would identify all the pages with those words on it and show you the ones that had the most links to them.

    As people searched and visited pages from the search results, Google would also track the pages that people would click to from the search page. Those that people clicked would also be identified as a better quality match for that set of search terms. If the person quickly came back to the search page and clicked another link, the match quality would be reduced.

    Now, Google is using natural language processing, a method of trying to guess what the user really wants. From that it it finds similar words that might give a better set of results based on searches done by millions of other people like you. It might assume that you really meant this other word instead of the word you used in your search terms. It might just give you matches in the list with those other words as well as the words you provided.

    It really all boils down to the fact that Google has been monitoring a lot of people doing searches for a very long time. It has a huge list of websites and search terms that have done the job for a lot of people.

    There are a lot of proprietary algorithms, but the real magic is that they’ve been watching you and everyone else for a very long time.

What programming language powers Google’s search engine core?

C++, mostly. There are little bits in other languages, but the core of both the indexing system and the serving system is C++.

How does Google handle the technical aspect of fuzzy matching? How is the index implemented for that?

  • With n-grams and word stemming. And correcting bad written words. N-grams for partial matching anything.

Use a ping service. Ping services can speed up your indexing process.

  1. Search Google for “pingmylinks”
  2. Click on the “add url” in the upper left corner.
  3. Submit your website and make sure to use all the submission tools and your site should be indexed within hours.

Our ranking algorithm simply doesn’t rank google.com highly for the query “search engine.” There is not a single, simple reason why this is the case. If I had to guess, I would say that people who type “search engine” into Google are usually looking for general information about search engines or about alternative search engines, and neither query is well-answered by listing google.com.

To be clear, we have never manually altered the search results for this (or any other) specific query.

When I tried the query “search engine” on Bing, the results were similar; bing.com was #5 and google.com was #6.

What is the search algorithm used by the Google search engine? What is its complexity?

The basic idea is using an inverted index. This means for each word keeping a list of documents on the web that contain it.

Responding to a query corresponds to retrieval of the matching documents (This is basically done by intersecting the lists for the corresponding query words), processing the documents (extracting quality signals corresponding to the doc, query pair), ranking the documents (using document quality signals like Page Rank and query signals and query/doc signals) then returning the top 10 documents.

Here are some tricks for doing the retrieval part efficiently:
– distribute the whole thing over thousands and thousands of machines
– do it in memory
– caching
– looking first at the query word with the shortest document list
– keeping the documents in the list in reverse PageRank order so that we can stop early once we find enough good quality matches
– keep lists for pairs of words that occur frequently together
– shard by document id, this way the load is somewhat evenly distributed and the intersection is done in parallel
– compress messages that are sent across the network
etc


Save 65% on select product(s) with promo code 65ZDS44X on Amazon.com

Jeff Dean in this great talk explains quite a few bits of the internal Google infrastructure. He mentions a few of the previous ideas in the talk.


He goes through the evolution of the Google Search Serving Design and through MapReduce while giving general advice about building large scale systems.

https://www.youtube.com/watch?v=modXC5IWTJI&t=30s
 
 

Here’s a link to his slides:

As for complexity, it’s pretty hard to analyze because of all the moving parts, but Jeff mentions that the the latency per query is about 0.2 s and that each query touches on average 1000 computers.

Is Google’s LaMDA conscious? A philosopher’s view (theconversation.com)

LaMDA is Google’s latest artificial intelligence (AI) chatbot. Blake Lemoine, a Google AI engineer, has claimed it is sentient. He’s been put on leave after publishing his conversations with LaMDA.

If Lemoine’s claims are true, it would be a milestone in the history of humankind and technological development.

Google strongly denies LaMDA has any sentient capacity.

Fun facts about Google Search Engine Competitors

r/dataisbeautiful - [OC] Google dominates the search market with a 91.9% market share

original post here


Build the skills that'll drive your salary into six figures

Data Source: statcounterGS

Tools Used: Excel & PowerPoint

Edit: Note that the data for Baidu/China is likely higher. How statcounterGS collects the data might understate # users from China.

Methodology

Baidu is popular in China, Yandex is popular in Russia.

Yandex is great for reverse image searches, google just can’t compete with yandex in that category.

Normal Google reverse search is a joke (except for finding a bigger version of a pic, it’s good for that), but Google Lens can be as good or sometimes better at finding similar images or locations than Yandex depending on the image type. Always good to try both, and also Bing can be decent sometimes. 


Bing has been profitable since 2015 even with less than 3% of the market share. So just imagine how much money Google is taking in.

Firstly: Yahoo, DuckDuckGo, Ecosia, etc. all use Bing to get their search results. Which means Bing’s usage is more than the 3% indicated.

Secondly: This graph shows overall market share (phones and PCs). But, search engines make most of their money on desktop searches due to more screen space for ads. And Bing’s market share on desktop is WAY bigger, its market share on phones is ~0%. It’s American desktop market share is 10-15%. That is where the money is.


What you are saying is in fact true though. We make trillions of web searches – which means even three percent market-share equals billions of hits and a ton of money.

I like duck duck go. And they have good privacy features. I just wish their maps were better because if I’m searching a local restaurant nothing is easier than google to transition from the search to the map to the webpage for the company. But for informative searches I think it gives a more objective, less curated return.

Use Ecosia and profits go to reforestation efforts!

Turns out people don’t care about their privacy, especially if it gets them results.

I recently switched to using brave browser and duck duck go and I basically can’t tell the difference in using Google and chrome.

The only times I’ve needed to use Google are for really specific searches where duck duck go doesn’t always seem to give the expected results. But for daily browsing it’s absolutely fine and far far better for privacy.

 

Phone screen shows text: LaMDA: our breakthrough conversation technology

  • Got a question what’s the likelihood of getting a virus/malware from opening a google drive link. (Useing iPhone)
    by /u/Gloomy-Rule-1854 (Google) on July 3, 2022 at 12:53 pm

    submitted by /u/Gloomy-Rule-1854 [link] [comments]

  • about Google lens
    by /u/Organic-Ad-503 (Google) on July 3, 2022 at 10:04 am

    Why doesn't Google lens show any results for NSFW images (hentai) I have turned off the safe search still Google lens doesn't show results submitted by /u/Organic-Ad-503 [link] [comments]

  • Google: E-ticaret Siteniz için 8 SEO İpucu — Dijital Pazarlama Notları
    by Mert Erkal (Google Search on Medium) on July 3, 2022 at 7:40 am

    Google: E-ticaret Siteniz için 8 SEO İpucuContinue reading on Medium »

  • Yahoo keeps replacing my browser
    by /u/PlagueDuck (Google) on July 2, 2022 at 7:39 pm

    If this isn’t the place for this then recommend another subreddit but yahoo keeps taking over my browser. I’ve done everything I could think of and what I saw online but it just comes back in a few days ☹️ submitted by /u/PlagueDuck [link] [comments]

  • WHERE IS THE SUPPORT MEGATHREAD?!
    by /u/Cerulean_MC (Google) on July 2, 2022 at 7:08 pm

    bruh submitted by /u/Cerulean_MC [link] [comments]

  • Vloguinho conversas entre Lanche da tarde 😀
    by /u/Firm-Reach-6387 (Google) on July 2, 2022 at 6:18 pm

    submitted by /u/Firm-Reach-6387 [link] [comments]

  • Google Merchandise Centre
    by /u/subwoofersmania (Google) on July 2, 2022 at 5:13 pm

    submitted by /u/subwoofersmania [link] [comments]

  • My rival manipulates google autocomplete and now whenever I write keywords on google his name appear
    by /u/scurtel (Google) on July 2, 2022 at 1:47 pm

    He found out to manipulate the Google autocomplete. He created his website this year and his website has very low da an pa. He has been a lawyer for a year. For example if you write Adana boşanma avukatı (means Adana divorce lawyer in Turkish) and google autocompletes his name below. He also did it for every key words. Adana boşanma avukatı (Adana divorce lawyer), Adana ceza avukatı (Adana criminal lawyer) and Adana icra avukatı( Adana car accident lawyer) . Google autocompletes his name below. All of these words are very important and business worthy. How did he do this? I reported him several times but Google did not do anything. What can I do? This is the biggest spam I have ever seen, he could destroy online business for lawyers in Adana with this spam. I think he uses bots and searches for exact keywords. Here are the screenshots of 3 keywords he did it you can check with yourself in my google help thread; https://support.google.com/websearch/thread/169569576?hl=en submitted by /u/scurtel [link] [comments]

  • No music provider options on Google Home app
    by /u/SovietCat90 (Google) on July 2, 2022 at 1:04 pm

    Hey, I recently got a google nest mini, and I tried connecting it to spotify but I don't get options in the Google Home App. (I tried getting help in r/googlehome but got no responses) submitted by /u/SovietCat90 [link] [comments]

  • Live caption and live transcribe.
    by /u/TastyEngineering (Google) on July 2, 2022 at 11:56 am

    I’ve been wondering, does anyone know why live caption happily continues to run when I hit mute, but live transcribe needs to use the microphone? Context. I’m not Deaf or HOH. I actually have super sensitive hearing, and hearing processing issues. Which means that live caption is a dream and live transcribe is a nightmare (especially when I have to turn the volume up for it to work). I’d settle for being able to copy paste segments from Caption. submitted by /u/TastyEngineering [link] [comments]

  • Interview for Google Data Center Technician
    by /u/jheyzen (Google) on July 2, 2022 at 10:54 am

    So I have an interview coming up for a DCT position. What sort of questions will I be asked, what can I do to prepare? Are there any coding questions for DCT positions, I have no coding background. But looking to learn and switch to a Computer Science program in the future. I am beyond nervous, any tips would be appreciated. submitted by /u/jheyzen [link] [comments]

  • Google Stadia Expanding to Mexico
    by /u/jdeslip (Google) on July 2, 2022 at 9:57 am

    ​ https://preview.redd.it/v35k7as7o4991.png?width=1920&format=png&auto=webp&s=18812196a18110fb6e5abe9c1c5abaff145e0fda At a Google for Mexico live event yesterday, Google announced it is expanding Stadia into Mexico. Mexico alone would bring Google Stadia to a territory with 130 million people. Increasing Stadia’s reach from 861 million to 991 million – a very significant 15% increase that is similar to their 2020 expansion in europe. More details: https://stadiadosage.com/stadia-mexico/ submitted by /u/jdeslip [link] [comments]

  • Finding Size Larger than Uploaded Image or Sorting Feature
    by /u/TechnicGeekOne (Google) on July 2, 2022 at 3:59 am

    Hello, I want to know if there is a way to find same image that I have uploaded but in higher resolutions. At least if there is a way to sort search results from highest to lowest resolution. Thanks. submitted by /u/TechnicGeekOne [link] [comments]

  • Google Chrome will soon let you translate selected text.
    by /u/Leopeva64-2 (Google) on July 2, 2022 at 2:59 am

    submitted by /u/Leopeva64-2 [link] [comments]

  • Ping pong game in iOS google app (go to tabs and swipe up a few times)
    by /u/starlightclient (Google) on July 1, 2022 at 11:43 pm

    submitted by /u/starlightclient [link] [comments]

  • Uma análise profunda da SERP: Como pensar SEO na era da Inteligência Artificial
    by Lucas Vieira (Google Search on Medium) on July 1, 2022 at 8:50 pm

    Em 2020 o Google publicou um artigo no qual foi feito um estudo profundo sobre o comportamento de compra das pessoas atualmente. Como…Continue reading on Medium »

  • What is content://com.android.browser.home/
    by /u/DroidBot1 (Google) on July 1, 2022 at 5:33 pm

    For Android browsers, the content://com.android.browser.home/ default command line is quite common. By tweaking the Android browser’s default settings, you can enhance your user experience using the command line. Want to know more about Android browsers and how to customize them? ​ content://com.android.browser.home/ What is content://com.android.browser.home/ Android offers a rich user interface, smooth performance, and a variety of Android versions and models that have made it the most popular mobile device of all time. The Play Store offers a variety of free apps and expandable memory. Using the “content://com.android.browser.home/” syntax, the default browser homepage can be customized. The default homepage is pre-installed on smartphones when they are purchased. It is possible to change the default homepage of any web browser if it does not meet your preferences. Android is an open-source operating system and anyone can customize the look and feel of the home screen to their liking. But still, there are some default features that come preinstalled on the phone, which can make it hard to customize the device according to your device. Depending on the user’s preferences, the default app settings and the default account can be changed. However, you will need additional commands and syntax to get to these special settings. Basically, this is the process of changing a device’s default browser’s homepage. Any Android device can be customized by this universal syntax. Keep reading the article and we will learn how to use the content://com.android.browser.home/ syntax and it’s useful for tweaking the Android Browser Homepage. Other content syntax for Android Browser: content://com.android.browser.home/ content://com.android.browser.home/index content://com.android.browser.home/index — google search content://com.android.browser.home/facebook content://com.android.browser.home/youtube content://com.android.browser.home/most_visit content://com.android.browser.home/google content://com.android.browser.home/bookmarks How to Use content://com.android.browser.home/- Homepage? Knowing what “content://com.android.browser.home/” means, let’s start using the syntax to configure the default browser on the Android device to change the home page. On Android devices, the default browser is called “internet” or “browser”. The default Android browser is instead preferred by a few users. Still, you will be able to get rid of Android’s annoying Home page that runs whenever you open the browser. Read Also: How to Fix CQATest App Causing Errors on Android Steps to Change Default Home Page on Android Browser. Firstly, you have to Open the Default Browser on your Android. In the Next Step, Tap the menu button available on the top-right side of the device screen. Now, Click on the menu i.e, ‘Settings’, and then move to the sub-menu until you find the ‘Android Browser Settings’ option. Click on it once you find it. Click the ‘General’ tab. Then, Click on ‘Set Homepage’ to complete the process. Simply enter the URL you wish to use as the default page or select ‘blank page’. If you do not wish to use the Default Android browser, you can insert the following URL into the Address bar when it opens: READ MORE: how to change the default home page of the Mozilla Firefox web browser easily! submitted by /u/DroidBot1 [link] [comments]

  • Won +48 play points
    by /u/Zanejuice (Google) on July 1, 2022 at 5:05 pm

    submitted by /u/Zanejuice [link] [comments]

  • So no one at google realised this, wow.
    by /u/WanaBeMillionare (Google) on July 1, 2022 at 5:07 am

    ​ https://preview.redd.it/mmf7zr9m3w891.png?width=644&format=png&auto=webp&s=695759479f923274589e35db570d3b540c0b9a97 submitted by /u/WanaBeMillionare [link] [comments]

  • Not seeing the 'support megathread' so here goes- Google Takeout
    by /u/SecretGaygent42 (Google) on July 1, 2022 at 4:06 am

    Hey, I am currently trying to complete a google takeout transfer. I did one before that didn't work (because of an error I made in asking it to transfer something I didn't want and that put me over the available storage space). My first time around it sent me an email within half an hour saying my transfer had started and to expect it to be done within a few hours to a week. The thing is, I started this current transfer hours ago and have seen no such email. There doesn't seem to be anything transferring into my drive or email, either. I'm not sure if I'm just being impatient or if I need to do something like try again. ​ edit: typo fixed submitted by /u/SecretGaygent42 [link] [comments]

  • Google Makes Deal With Developers in Latest App Store Shift
    by /u/Metanism_ (Google) on July 1, 2022 at 2:29 am

    Google reached an agreement with US developers that will let consumers subscribe to services outside the company’s Play Store, marking the latest shift for an app-store economy that it dominates alongside Apple Inc. The Alphabet Inc. division also plans to provide $90 million to support developers who earned $2 million or less annually through the company’s app store from 2016 to 2021. Google will continue to charge a 15% fee on the first $1 million in annual revenue earned from the Play Store by US developers, the company said Thursday in a statement. submitted by /u/Metanism_ [link] [comments]

  • Google Merchandise Store Online
    by /u/subwoofersmania (Google) on June 30, 2022 at 10:44 pm

    submitted by /u/subwoofersmania [link] [comments]

  • Google Search Hacks for Event Planners
    by AllEvents.in (Google Search on Medium) on June 30, 2022 at 9:48 am

    The event industry is growing like inflation these days 🔼!Continue reading on Medium »

  • Pixel 4a unlimited storage saver worst quality compressing
    by /u/Common_Luck2699 (Google) on June 30, 2022 at 8:37 am

    its stated that google pixel 4a has unlimited storage saver space but since when i uploaded my wedding album and created a shared album to share the photos with the relatives, the photo was actually 12 mb only but it got compressed to 360kb. HOW TO PREVENT THIS FROM HAPPENING ? IF GOOGLE IS STATING THAT THE FILES WILL BE CONVERTED TO 16MB THEN WHY IS PHOTOS WITH 20 OR 16 OR 12 MB CONVERTED INTO WORST QUALITY THAT IS KBS.......................................... submitted by /u/Common_Luck2699 [link] [comments]

  • google fix this hiding your problematic ads from your report tool.
    by /u/Evilskydiver270 (Google) on June 30, 2022 at 8:09 am

    submitted by /u/Evilskydiver270 [link] [comments]

  • Google Drive Transfer Stuck "In Progress"
    by /u/ooh_daddy (Google) on June 30, 2022 at 3:57 am

    Hey so I'm transferring my old Google Drive (that is being disabled) to my personal and for some reason the transfer is stuck "in progress?" I am using Google Takeout but it has been stuck like this for the past 5 days. Anyone know about why this might be the case? https://preview.redd.it/t5y4woq3mo891.png?width=574&format=png&auto=webp&s=90fc2b7a5aeaf3f0df311599b0fbafcd35a220a2 submitted by /u/ooh_daddy [link] [comments]

  • How to remove contact suggestions from maps location sharing
    by /u/arnoldi (Google) on June 30, 2022 at 2:51 am

    I would love to remove a contact from the recommendations bar in the Google maps location sharing flow. I've removed them from my Google Contacts and Other Contacts, but they are still suggested. I think maps must have its own list of contacts. Seeing this contact is a distressing experience, and I'd love any help. Thank you. submitted by /u/arnoldi [link] [comments]

  • cssi
    by /u/droopy-peter (Google) on June 29, 2022 at 11:44 pm

    hello im doing cssi this year and was wondering if anyone who did it in prior years have kept in contact with google and or have had job offers because of cssi? thanks! submitted by /u/droopy-peter [link] [comments]

  • The World’s Top 10 Search Engines
    by Typing Studio (Google Search on Medium) on June 29, 2022 at 12:04 am

    The top 10 search engines are listed below.Continue reading on Medium »

  • Lava Woman
    by Zach Thom (Google Search on Medium) on June 28, 2022 at 7:02 pm

    Chapter 1Continue reading on Medium »

  • Because you can feel Google’s AI
    by Jorge Cimarrón (Google Search on Medium) on June 27, 2022 at 4:12 pm

    ¿Puede “Pensar” y “Sentir” Emociones una Máquina?Continue reading on Medium »

  • Setup Your SEO Toolbox
    by Jamie Wen (Google Search on Medium) on June 27, 2022 at 12:13 pm

    A list of SEO tools for web developers that will help you work more effectivelyContinue reading on Medium »

  • Five steps to mastering keyword research
    by Digital First Aid (Google Search on Medium) on June 27, 2022 at 1:44 am

    Keyword research is the lifeblood of your business. Find out how to master SEO, Google rankings and customer search terms. Read on.Continue reading on Medium »

  • Organik Trafiğinizi Nitelikli Potansiyel Müşterilere Dönüştürmek İçin 12 İpucu — Dijital Pazarlama…
    by Mert Erkal (Google Search on Medium) on June 26, 2022 at 10:35 am

    Organik Trafiğinizi Nitelikli Potansiyel Müşterilere Dönüştürmek İçin 12 İpucuContinue reading on Medium »

error: Content is protected !!