You skipped lunch to come here about search, no pressure, so my name is Maria and I’m a webmaster trends. Analyst coming all the way from Switzerland to talk to you about search and I’m John Mueller. I’m also webmaster trends, analyst we’re both from Google Switzerland. Our role at Google is to help the web ecosystem to be more successful in Google search.
So we hope we can bring some of this across here as well great, so I want to start us off with an example of how this actually works, so you can get an idea of what you as a developer, can do to be more successful with search. So this is Japanese website Rakuten recipes and they wanted to get more users. They also have a ton of delicious recipes on their site, so they were wondering. What can we do to get more people to notice us in search and what they decided to do? Is they changed their CMS in order to be able to markup every recipe that is added to the site, with structured data markup? What this does is it lets the search engine know that there are different entities, so things like cooking time, different ingredients, a picture and lets us display the result in a much more attractive way in the search results and the site itself was in Japanese.
So we decided to switch off the markup for those of you who don’t speak, Japanese to something more legible, so instead of dumpling, so you have here a party coffee cake, but it works the same way for dumplings, and this is how this ends up. Looking like in the search results, so you can see that, in addition to the regular elements of the search page, so they have a title, they have the URL and then they have the description.
They also have a really nice picture of dumplings, and then they have the cooking time as well. So this worked out for them pretty well in fact kind of spectacularly, so they got 2.7 times more traffic from search yeah and we thought you know developers usually don’t get as much advice around search and they might not even know about all of the pitfalls and Possibilities shouldn’t we share some of the knowledge with them, so with you all as well, so we hope we can show how search can make your projects a little bit more successful.
So today we’ll look into various types of public web presences and gives you a broad overview of specific details that you, as a developer, can read out for and implement, and these details can help make your projects more successful, on search making it easier for search engines To send users to your projects directly, so you might be thinking as a developer. I don’t really care how or if my stuff appears in search, but probably your customers, your users, the people that you’re building these projects for they do care.
And since you control how your content appears in search, you can have a huge impact here, think back about Rakuten recipes and what they did there. Globally. Google search and Google News send Philly ins of visitors each month to web sites for free, and it’s not just about web sites. We’re going to be looking into various ways that search can work depending on what you’re working on a website is just the most common format.
You could also be building a web app which is kind of similar, but slightly different as well or you could be contributing to a content management system, its so-called CMS, which enables others to build web sites of their own or as a part of that, maybe you’re Working on plugins or themes or extensions for these content management systems, we’ll take a brief look at these, as well as detailed recommendations for each of them and, as I mentioned before, search brings billions of visitors to websites every month.
That’s a lot of visitors. We serve trillions of searches each year and out of those, this is quite surprising for me every time I look at it about 15 % of the queries every day are completely new ones. Things we’ve never seen before, so maybe they’re looking for one of your projects and regardless of what you’re building if search engines, understand your content, you could get a lot more visitors and potential customers with search.
So you as a developer, you can control that through the way that you set up your website or content platform, so under these Sandy’s better, let’s take a quick look at how search works right so in order to be successful as a developer in search, you need To know at least the basics of how it works, and I’m going to take you through the super super high-level picture of how it works. If you’re interested in the details, google.
Com slash jobs, welcome to apply, and then we can go into a lot more detail. But let’s get started with the super high level picture, so we generally talk about three things. First crawling and discovery, then indexing and finally ranking and serving so I’m going to show you very briefly what each of these things is about. So, of course, in order for us to be able to show anything in the search results.
We have another set of systems which are known as schedulers. So they queue the URLs for the crawlers to go and fetch, and all of this gets then stored. So you might think that this is a pretty simple process. But if you start thinking that we have to do this 20 billion times per day, then you kind of get an idea. It’s a little bit trickier than it seems at first sight. In fact, in 2016 we saw a hundred and thirty trillion pages and a new link that we see usually there’s two more links that we’ve never seen before so there’s constantly new stuff and we have to decide what to crawl.
How to update and to do this. In the most efficient manner, so whilst we find the content, we have a series of other tasks. First, we have to make sure that we are allowed to access that content, and for that we will first go every time. We access a site we’ll go to a file called robots.Txt, which is a pretty simple file containing instructions to search engines and other crawlers, and it tells you this is okay to fetch, and this is not okay and we obey this very strictly.
So that’s the first thing that we’ll try to find on a website. The other thing that we’ll try to do is to get as much content as possible without troubling the normal work of the server. So the website can function and serve its clients as usual and then finally we’ll try to handle errors gracefully. So as a developer, you have two tasks here: the first, if you remember again that we do fetches 20 billion times a day and we see trillions of pages every year – is that your content should be really easy to discover so ways to do.
That and John will go into a little bit more detail later is to, for example, submit to us a list of URLs. They have like a sitemap or also check that all the resources that are necessary for your site to be rendered are accessible to our crawlers. So once we fetched everything that we were able to fetch, we go to the next stage, and that is indexing. So here we’re going to parse the content and into this comes things like what language is this page? Are there any images? Is there a title? Is there a description and other different elements on the page, so to do that, we also try to render the page and as a developer, especially if you’re building a lot of really cutting-edge fancy things.
You have to keep in mind that currently, the search systems are using chrome 41 to render pages. So not all of the different functionalities that you might be thinking about could be supported by the search rendering systems, and if you want to find out more, I would suggest that you have a look at the talk that John did earlier today in the morning. In case you didn’t wake up at 8:30 to see it, it will be available in YouTube and you’ll be able to see a lot more about what we support and search and how to render things properly.
So, of course, given the huge amount of pages on the web, we also don’t want to index more than one of each unique thing. So we have a lot of systems in place to eliminate duplicates and to keep only one copy of each thing and then finally, we don’t want error pages and we also don’t want any spam. So we will kick all of that out and everything else that we want to keep we put in the index and we process it so that it’s ready to serve to users when they search.
So for you as a developer. Here I guess it’s important to remember that key elements like titles and descriptions are available in each page that your users are creating and then also to check how it’s rendered. But John will go into a lot more detail here later and then. Finally, once we have everything in the index, when users start searching we’re going to pull a set of pages, that we think are relevant results, we’re going to add a bunch of information that we’ve already accumulated to them like how important they are and how they relate To the users query and then we’re going to show them in some specific order that we think it’s most relevant for this user.
So this is mostly on our side and you don’t need to worry about anything here if your content is already accessible and easy to render, but if you’re really interested in ranking and search quality again, Google Chrome, slash jobs, there’s plenty of interesting problems to solve. So now that you know how search works, let’s have a summary of the two things that you need to remember. There’s. First, you have to help us find the content and second, you have to help us evaluate the content.
So if you’re able to do these two things you’re pretty much set as a developer. Now this is super super high level. So what we’re going to do next is show you how you can apply this for each specific thing that you need to build, so we’re going to start with websites and John is going to share with you some very specific advice about what to do, and what Not to do when you’re building a website for someone all right thanks Maria, that was a great introduction into search.
So, like you said, let’s start with websites, you can build a maintained one for yourself to kind of showcase, your own content, or maybe you’re doing that for other people to show to let them create websites on their own. You might be thinking that showing up in search isn’t really your job as a developer, but, like we mentioned before as a developer, you play a really big role in kind of putting everything in place, so that search can pick up the content properly.
So that’s really important for us when it comes to websites. I think it’s worth taking a really big step back and looking at the absolute basics. So for us, that’s a URL. That’s essentially the address. That’s used to address a single piece of content on the web. Perhaps surprisingly, URLs are the cause of an solution to a lot of web search problems. Traditionally, URLs on the web started out quite simple.
That’s usually no big deal and you can look at that. You think well, index.Html. That’s obviously homepage I mean that’s like every developer knows that, but for search engines, that’s not so obvious. It could be something completely different. Sometimes you also just track ad tracking parameters to URLs, and all of these different URLs are essentially for search engines, separate pages that we could look at and say well, there might be something different here and you can imagine at 20 billion times a day that could Lead to a lot of inefficient crawling, so we prefer to have a single URL per piece of content, and there are two ways that you can do that the first is to consistently use the same URLs across your whole website.
So if you have internal navigation link to the same pages, if you have a sitemap file, like Maria mentioned, you use the same URLs there. If you use anything to guide people to your websites, make sure you use the same URLs there. Instead of having these different patterns that only two the same thing, and secondly, one element that you can also use is the rel canonical link element, which is something you can place in the head of a page that tells us well search engines or Google.
If you look at this page, this is actually the URL that I prefer you look at. This is the one that I want you to index and together. This makes it a little bit easier for search engines to pick the right URL. So we have your ELLs covered. What else is there? Let’s take a look at a typical search results page, so we have on top the title in this case the Google i/o schedule page. Then we have the URL which is right below it.
In this case, it’s a it’s a breadcrumb URL, we’ll look at that briefly a bit later as well, and then you have the description. So these are three elements on a search results, page that are immediately visible to everyone who is searching for something and they come from your pages directly. So it’s something that, as a developer, it’s really easy to place. When you look at an HTML page, they’re, very visible and easy to find, so we have the title on top.
This is really easy to put in here. We have the canonical tag, the rel canonical link element, which is also really easy to place, and we have the description meta description. So while these elements don’t directly affect the ranking, so the order that Maria talked about they do affect how we show a page in the search results and with that they do affect how people actually come and visit your pages or not.
So we’ve seen a few of the basic elements here, like the metadata, the titles URLs and the descriptions. What could you as a developer do to make that a little bit easier, especially if you have various people who are using your website your project, to put content online? We recommend making it as easy as possible for them to do the right thing, so not just for you as a developer, to put titles descriptions and all of that into your pages, but also for those who are creating pages on your platform.
To put that in there, so here you see a user interface from blogger with a really easy way to just add a description to individual pages, and we feel the easier it is for for people making your pages to actually put this content in there. The more likely they’ll actually do that. So when we looked at the search results, we saw this kind of breadcrumb there as well, and a breadcrumb is for us something.
That is something that you can provide on your pages to make it easier to understand where this page belongs within your website. We call this a type of a rich result, because it’s not just the pure text text result and there are different kinds of rich results that you can also use. For example, you could add markup for articles. If you have articles on a page, you could tell us about podcasts, which is really cool, because there’s a podcast player built into the search results.
So if you have a podcast, if you have a project that includes audio content, then suddenly that content is immediately available. In the search results, without anyone needing to install an extra app which is really cool and then finally, recipes, of course, which we saw with Rakuten in the beginning. So how do you get all of these rich results? Well, Maria mentioned that briefly. Essentially, it’s just a bunch of json-ld markup that you can add to the top of your pages.
That gives us a lot more information. So this is something that you can just add to the pages here. It’s really easy to add. We have a bunch of different types of markup that you can add here, there’s a code lab here for i/o as well on adding structured data markup. So if you’re curious on how to do that, I definitely take a look at the code lab. I have a link here and the code lab includes information on finding the right types of markup to add how to add it and how to test it.
So that’s a great thing to check out another element when it comes to web pages. If you’re working on just a general web page project is speed. For us, speed is a ranking factor at Google, so it helps us to determine which pages we should show in. In the order in the search results, but generally we’ve also found that speed. It makes a big difference even outside of search engines and the various tools to test speed.
We we have a link here that gives you an overview of the different testing tools that we have. One of the tools is PageSpeed insights, which I showed here. That gives you a great overview of what you could be testing, what you could be looking at what you could be improving and then one other really important tool when it comes to search, is search, console kind of what the name says so within search console, you Get a lot of information about this whole pipeline that Maria showed everything from discovery to crawling to indexing and to serving so how we show your pages in the search results.
You can find information about this in search console. Additionally, we’ll also alert you of critical issues. As they arise, so we strongly recommend that everyone checks this out if you’re, making a public web presence, anything that you want to have indexed and searched and looks like a lot of you do so. The first step when it comes to search console is to verify ownership. We don’t show the data in search console to just everyone.
You have to kind of prove to us that this is actually your website. One thing that I find really important here is, if you’re, making a project for others online, make it as easy as possible for them to verify owners so make it possible for them to add any of these verification tokens so that they don’t always have to go Back to the development team say hey, I need this special file that has this content and put that on a page.
So we we talked about websites quite a bit, but web app is another really important topic which I imagine a lot of. You have seen in different ways here at i/o already for us. A web app is kind of like a normal website, but it provides a lot more interactive functionality. Interaction may be logged in view personalization. Maybe it has parts that don’t actually need to be indexed as well. For example, a travel business might have information about timetables and general pricing, but also have detailed information about kind of specific connection plans for individual connections or personalized pricing as well or in this case.
That, generally, don’t play such a big role on websites in general, especially if you’re making normal HTML pages. The first one is how to actually find URLs on your site. So we talked about URLs briefly Maria mentioned how important they are for discovering pages and within web apps. We’ve seen that people sometimes don’t use traditional anchor tags to. Let us know about your ELLs, so in particular we we love finding things like this.
So there are different things that you can do here to. Let us know about this. The first is obviously to avoid actually going off and crawling all of these different URLs. So if these don’t provide unique functionality that you need to have indexed separately, maybe you can use other ways of linking to them other than a element. Another thing that you can do is within search console tell us about individual parameters within the URL that you don’t care about.
This is really important for us because we’re switching to mobile first indexing, where Googlebot is actually using a mobile device to for all pages rather than a desktop device. So definitely make sense to check this out and we also have a bunch of best practices and general guidelines that that apply more to web apps that you can check out in the other session as well. So what do you do if you’re not just building one application or one website, but rather a whole platform? I don’t know Maria.
Can you tell us more? I have some ideas all right, so you could be building an individual site or a web app for someone or for yourself or you could be contributing to an entire content management system or another hosting platform. And here what I mean by this is any type of platform where other people can create their own online presence, so it can come in different flavors, for example, it could be something like WordPress, where you could download it and host it on your own server or It could be a fully hosted system, plus your own domain, like Squarespace, for example, or it could be something where you just get a URL on their own domain and also it’s hosted by them like Tumblr.
So there are all these different flavors and you could be working for a system like this which, in its own turn, has a bunch of users. So what you do affects all of these people, and that is a lot of power and a lot of responsibility. So we’re going to talk about what can you do to make all these people successful in search by making some changes to the platform itself? And this is a really important topic for us right now, because more than 50 % and growing of the web is currently build on various CMS’s.
So more than half of the content on the web is affected by the systems and if you’re working on one of them were planning to do so in the future, it’s really great, if you’re able to make those people successful in search as well, because that’s why They came on the web, they wanted to connect to others, maybe find some customers and so forth. So we’ve been thinking a lot about this and we’ve built a set of api’s to help you integrate search functionality directly into the interface of those systems, and I want to show you api’s and how they’ve been integrated already.
Maybe, to give you some inspiration and some ideas about what you can do so as John was mentioning before the first thing that we need in order to show any type of search, information or search functionality is to have proof that you are indeed the owner of The site – and he mentioned how this works for individual sites, so you can have an HTML file. You can use a DNS entry and so forth, but for those users, especially for the less savvy CMS users – wouldn’t it be great if you could actually simplify it to one click and it is possible with the verification, API and three-legged OAuth.
So we’ve built this API. So that you can use it, and if the user authorizes you, you could verify their site, which is hosted on your platform on their behalf. They just need to click one button and they immediately have access to all the search information. So the experience for them is really smooth, and you can do this for a thousand users or for two million users or whatever it is, and then immediately they get access to all kind of interesting stats.
Which brings me to the next API, which is the search console API and that provides access to aggregated stats per site. So you can see things like clicks impressions, crawl errors. You can submit a sitemap through there and you can slice and dice this in many different ways. So, for example, you can look per country per time period or per device, and you can build very interesting interfaces with that, and here on.
The slide is an example of a request where you just pulled the top 10 queries per clicks for a specific period of time and then, as a result, you would get a table where the query clicks, impressions position and so forth. Now a table in itself might be informative, but it’s not really exciting. So let me show you some ways in which existing CMS’s have actually integrated. This we’ve been working with Wix and they created this achievements sidebar for their users, so they’re using the search analytics data to give this little badges every time.
Something happens that they think the user will be happy to hear about. So their users are super excited about this gamified approach and they’re constantly looking there in order to see okay. What did I get now? What did I get now? So here’s clicks and impressions built in an achievement sidebar like this we’ve, also been working with Squarespace and actually just this Monday, they announced this new report that they integrate it into the interface of their own CMS.
So what you see here is the one-click verification when the user click connects to Google in the backend, their site gets verified, and then this report gets populated with information from search console. So here the user can see clicks impressions and the time series over the last month. Squarespace has a bunch of other analytics reports inside their CMS, so people can compare and build the full picture of how they’re doing in search – and at this point they don’t even know that search console exists, but they have everything that they need to know how they’re Doing and to accomplish the correct tasks right there in their Squarespace dashboard.
So we’re pretty excited about this kind of functionality and we want to build up on it and we would be looking forward to work with other CMS’s if you’re, representing one and you’re interested in this. Another thing that we really wanted to help users with is get their content as fast as possible on the search results, and so we’ve been looking into ways to use the indexing API that we have in order to get content submitted super quickly and then also be Able to share the indexing decisions, so what did our search systems think about the CRL and what do they want to do with it? And we worked on this for a few months and at this point is in a place where this can happen within seconds.
So again, with Wix, we built a pretty cool integration where, when a user submits a page and it matches a certain quality criteria, basically they can click a button within the Wix interface and then the page gets submitted through the indexing API and then, after that, they Immediately get a response if their page got on the search results or not so for the Wix users. This is a pretty cool experience because they can see their page in the search results immediately after they’ve created it there’s no waiting, there’s no wondering and my own search or not within seconds they’re on Google, so we’re interested in working with other CMS’s and if you Represent some kind of content management system or a platform which lets users create their own presence online and especially if your users are less savvy and they don’t really know what to do with search.
We are really interested in talking to you to see if it might be a good fit to participate in the CMS Partnership Program. So, there’s a link on the slide which will take you to a forum and there you can tell us a little bit more about who you represent and how you would like to work with us. So, looking forward to hearing from some of you hopefully now you could be contributing not just to the core product of the CMS, but to a bunch of other things which people install in order to enhance the functionality of their site.
And one of those things are plugins now plugging here is defined as any kind of add-on that people would add to their site. So, for example, things like a shopping, cart or maybe a way to add reviews or a comment plug-in things like that. So while it can enhance the functionality of the site, it can also significantly alter the functionality of the site in terms of performance and other factors. So I wanted to give you a few tips on what to do if you’re building plugins.
First of all make sure that it doesn’t slow down the performance of the site. So, in order to do this, have a test site, install the plug-in and then use our speed tools to make sure that the site with the plug-in is doing just as well as the site without the plugin. This is webpagetest.Org, which is one of the performance tools that we have, and the neat part about it is that it will give you a super, detailed breakdown, a float loaded and when so, you can see how your plugin is affecting the performance of this site.
So test that out, then, if you’re building a comment plug-in and if you been on the Internet in general, you will know that there are a lot of comments out there, which are maybe a little bit less valuable than other comments and especially in some cases there Altogether, spammy or there bots, which are going around and posting auto-generated stuff in order to create links that they’re hoping search engines will follow to some spammy websites.
This is not pleasant for any user and you can actually help them out a little bit if you’re, building a plugin like this by adding a specific type of annotation to those links by default, so that search engines know not to trust them. This is a link attribute that we call nofollow and basically, what it does is. It will just tell the search engines. Don’t follow this link, don’t trust it. So, if you’re building a comment plug-in, definitely consider adding this to the links in the comments.
Finally and kind of most, unfortunately, we’ve noticed that one of the main vectors for attack on websites and attacks on websites are increasing is hacks through plugins. So a lot of hackers and malicious other malicious people will get access to a site through an old plugin and if you’re, building plugins there’s a few things, you can do to make sure that your users are not affected by this. First of all make sure that every time you add an update, everybody who has this plug-in automatically receives it then make sure to follow coding, best practices so that there’s no backdoors that the hackers can exploit.
And finally, if you get tired of this plug-in and decide not to support it anymore, make sure it’s clear to people that this is not supported, so they don’t go ahead and install something that is actually making their site more vulnerable themes are another thing that is Very closely related to CMS’s and a lot of people install them and though, in order to improve the appearance of their scientific Euler or give it a specific like feel, so they can change how the site looks on how users perceive it.
But they can also really affect performance and they can also affect mobile friendliness. So again, here test your theme make sure that it’s responsive. You can do this with the mobile-friendly test that John was showing earlier and for performance. Specifically, we recommend again having a test site and then, with the theme and with another theme, having a look at how it performs lighthouse is one of our speed tools, which is really useful in this case, because it’s in the browser you don’t need to have have The site process, by search in order to test it so here’s a blogger site, the we use for purposes of this example.
We install the theme and then we use lighthouse to do the performance testing. So you see how long it takes actually until what they call the first meaningful paint, which is when all the elements appear to the user. So the overall score for this team was not super great and also the specific user metrics with we’re not great either. But then we went ahead and we switched to another theme, so you can see here it’s much much faster to load and the user can interact with it much faster as well and consequentially.
The score is also great. So if you’re building themes definitely make sure that it’s performant and also that it’s responsive by using the free tools that we provide so there’s a lot of stuff that we cover today and hopefully for any of these things that we’ve told you that you might be Building you now have enough tools and equipment to go ahead and make improvements and make your users happy and more successful in search.
We know that there’s many many different details and links that we provided. So if you have to remember just four things: pay attention to what John is going to tell you right now, all right thanks wow, that was a lot. Ok, look looking back at these things! There are a few common elements that that we covered that came up again and again. So first is remember the basics, URLs titles and descriptions they do matter.
They do play a big role when it comes to search. They play a big role in how people come to your site through search. Secondly, remember to take advantage of structured data like the Rakuten example in the beginning. Obviously, they saw a big change in the traffic from search even without ranking changes just by making their search results. Look a lot more visually appealing and then take advantage of all of the tools and api’s that we have available so use.
The search console understand how search console works, use the api’s from search console to make it better for your users, people who are using your products to really create fantastic web presences and then fine, especially if you’re, making something for other people to create web presences. In make it as easy as possible for them to do the right thing as well, so make it easy for them to put the right fields in in to add data about titles and descriptions on pages, make it easy for them to really create high performance.
Web pages, so these are only some general tips I think, to get started with they’re. Obviously, a lot of different aspects that come into play with search, but we think these are aspects are really critical to start with, and we have a lot more information in our developer Center developer guides. We have a search console Help Center with a lot of more information about search in general and specifically about search console.
If you have any more questions today will be in the web, and payment sandbox area later today, so feel free to come by there as well and finally, there of course other ways to reach out to us online as well. So you can find us on Twitter. We do live office hours hangouts, where you can join us with a YouTube, live hangout, we’re available in the webmaster help forum. If you have any questions, so don’t don’t let questions kind of stick around make sure you get answers to them from from us.
If you need them, we hope you found this introduction into search interesting. Thank you all for coming. We wish you and your projects more success online through Google search. Thank you.
Who is helping with your digital business footprint?