Author: Jimmy Dagger

If you are here, you are probably interested in communicating science and in writing a blog post about science. Our webinar will take 30 to 40 minutes. If you don’t have the time to read the article, you can also access the slide through the link below, in the youtube description, also other useful links that will be presented in the short course, and you cannot click in the article you can access in this description Before we start with the content of the course we want to shortly present ourselves, we are all early career centers who are involved in egu division blogs me, clara and violin.

We are involved in the ego. Cryosphere blog valeria is the main editor of the etu natural hazards. Blog and elenora is the main editor of the etiotectonics and structural geology. Blog eleonora cannot join us today, unfortunately, but she was strongly involved in preparing the content for this short course. As I already said, we are not fully we’re not working hundred percent on the blog. This is something we do on the side of scientific research, but this is maybe not the main interest of this course.

What you’re interested in is more what we’re going to do so in the next 30 to 40 minutes. We want to share with you some experience we had in running the science or the edu division blogs to help you draft a blog post. We will start with thinking about the motivation behind writing a blog post and the topic you want to share. Then we will think um about. Is there a best moment to write a blog post and where to po, to publish your post? So you want to do it on a personal blog or join a collaborative blogging platform then also you’ll have to think about who you want to reach with this blog post and finally, a big chunk of the short course will be about actually writing a blog post.

This was supposed to be a short course um, a participatory short course at edu. Now it’s a pre-recorded webinar, but we still encourage you to actively participate and to do so, we have prepared some practical exercises that you can do throughout the webinar, where, whenever you found the symbol shown here, you can then pause the article to work on these exercises And if you have done them all at the end of the webinar, you will have a draft for your own blog post.

Please take your time. Do it at your own pace? If you think some exercises are not interesting to you, then just don’t do them um. If you have a draft at the end, you can send us the draft for feedback, and we can help you get it published on the e2 division blogs if you’re interested, if you’re not interested at all in the practical part, just read the webinar without breaks and It will give you some inspiration, so, let’s start about thinking about your blog post, and the first thing you should ask yourself is: why do you want to blog so think about your aim? It could be advertise your own research to your peers, or you want to share your research or science in general, with more than just your peers, a larger audience, or you want to get a new perspective on an interesting subject, and you want to share this as Well to other people, or you want to network, because writing a blog post will make yourself visible to the rest of the community or even a larger public, and you will get to know other bloggers.

And finally, the aim – probably behind all of these is also communicating about science – is interesting to you. Now comes directly our first exercise, where you can think about the reason why you decided to follow this webinar. Why are you interested in science blogging? What would you like to achieve by writing a blog post? What is your aim today? Once you have your aim, you can also think about the topic very rapidly, because often it is very closely related to your aim.

For example, if you wanted to share your research with other people, then topic will be your own research. Then, of course you have to think about what parts of your own research you want to share. Maybe also you want to write about a particularly interesting publication, or you just want to. If your aim was to share about science in general, you could write about basic scientific principles or scientific fun facts and, finally, you can also share experiences from your life.

As a scientist such as field work or career, advices or advices for mental health, here again take a bit of your time to think about the topic. Do you already have a topic in mind? If not take some minutes to think of a topic you want to write about today. Now that you have your aim and your topic, let’s think about. When should you start to write um? There is not really a rule about this uh. We would advise just right when you feel inspired and motivated and of course, if you have time otherwise, there are not really limits.

You can write a blog post from the beginning to the end of your scientific career. You can write about your research in the beginning, so the motivation behind it or when the publication is ready there. We would advise to write when the publication is ready, because then the results are peer-reviewed, but that’s up to you and then of course, it’s also nice to share about a particular field campaign or experience.

So it also depends on the aim and the topic when you want to write. Also, the publishing is a matter of timing, usually just publish after having gotten feedback that improve your piece, but if there is, if your topic fits a current news context very well, you can also try to aim for uh this timely way of publishing. For example, uh. I’r involved in the etu cryosphere blog and i’m a ci scientist and every year in september, the arctic cis reaches its minimum extent and then we it’s kind of natural, to write a blog post about sea ice.

At that point, okay, and so now you approximately know what you want to write about and when now is uh. You also need the question of where um, because you can either publish on a personal blog on a collaborative blog. The advantages of a personal blog is that you have complete freedom, because it’s your own, it will reflect your main interest and it will be advertisement for your person, so it will be all about you, but of course you have to keep in mind that it is Also, very time consuming because you will have to take care of the website.

Take care of the structure. Do everything by yourself. While if you join a collaborative blog, the structure will be there already. You will have a platform to network, discuss and have feedback on your pieces. You have more visibility from the start, because maybe the blog is already famous or reaches already a lot of people, and you have to invest less time, because even if you want to post regularly um, you can do so with with much more time, because other people Will post as well but keep in mind also here that this is a team effort and that not everything will go um after what you want, so conflicts may originate in this webinar.

We will go through principles that are applicable to both structures, but um, keep in mind that we might be a bit biased towards the collaborative block. Because that’s our background, and now we also want to quickly present to you the egu division blocks, because we have been mentioning them several times already and you might not know exactly what we’re talking about. So they are what we call a collaborative blog platform.

Each year provides a possibility for each division to have a blog, and what you can see here are the divisions who already have blogs, and these blogs are run by volunteers, often early career scientists and the pub. And although the publishing platform is provided by edu, you still have much freedom for the content as long as the chief editors check it. It is based on a few regular contributors, but they also have scientific research to do so.

They are very happy if guest authors take up the task of publishing blog posts once in a while as well. If you are very interested about the ego division blogs, we also have a display in another session, where you can see some statistics about the audience and the posting ratio of some of the egu division blocks as an exercise. Here we invite you to navigate the ego. Blog universe to find inspiration, and maybe a blog that you feel your topic could fit in.

If you want to go the collaborative option, and now I give to valeria who will continue all right and after i’ve thought about the aim, the topic and when and where you want to publish and something important to keep in mind is who do you want? Your blog to be read by who, can you help the most with what you want to write so your target audience, and that could be the brother community of peers if you want to maybe blog about your main research topic or something very specific that can be Of interest for for your community, but also the general public, if you want to block of science in general or some basic principles that you would like to explain, or why not a specific group or community, let’s take an example of an earthquake that happens in a Specific region of the planet – and you would like you – are an earthquake um as a researcher, so you are a seismologist and you would like to explain a little bit better.

What happens? What happened there and what are the basics behind like earthquake concert, so you may want to address this topic for exactly a specific community in that region. But if it’s not only important for you to know what is your target audience uh? It is important as well to know how to be found by your target audience. An easy way to find blog posts is generally to browse the internet and use keywords and keep in mind as keywords, because they will come later again when we talk more about how to structure blog posts.

So what are readers potentially looking for? So what are the hot topics or the most research uh questions uh on internet so depending once again on your target audience? So if you’re your peers, maybe you want to check out the research gate, a question area for maybe hot questions around your discipline uh. While if you want to target the more general public, why not check out what is aston quora, for example, still related to your discipline or your field? And another important point is definitely the language that you decide to use in your blog post, english or your mother tongue.

Of course, if that is different from english – and this will depend on your name on your topic on your target audience as well as the blog style, for example, the ego division blocks, we all write in english more generally. Actually always so, you have chosen a topic. You like to write about by now and who do you want your post to be read by just think about it as this exercise and if it’s not only important who is going to read your blog for your blog post, it’s also important who’s going to understand.

You jargon: let’s talk a little bit about jargon, because it’s very important so jargon is any word or phrase that we use in our particular field or profession that loses or completely changes the meaning when you use it with the people outside your field. Some examples dating so, if you are a geoscientist for you dating, is probably determining the age of your sample, maybe a rock sample but more commonly.

This is the initial stage of a romantic relationship, so careful or stress generally, it’s a mental or an emotional strain, but scientifically it can mean a pressure, tension or exerted on an object. What about bar? I mean this is the unit of measure of atmospheric pressure, but don’t you usually go there for having drinks as well? So why don’t you think about one word in your field that may be misinterpreted or not understandable to the general public and try to find a more accessible alternative, so try really to use a specialized language only for a specialized audience, replace jargon when possible, with a Simple explanation of the concept or the process, and if you can’t avoid to use jargon, please make sure that you accompany that word with a simple example, but at the same time do not talk down your audience.

They are still knowledgeable people. So, using the same jargon, word that you choose in the previous exercise now try to substitute it with the simple explanation of the concept behind it, and now we go towards the structure of an actual blog post with blm. Okay. So now that, now that you have chosen both the topic and the target audience of your future blog post, the next step is to define how you want to present your content to the reader or, more specifically, to define the format of your future blog post.

So in the next few slides to help you do that, we will show you some examples of typical formats that can be used for a blog post. So a first option, if you decide to write a blog, is to write what we called here. A quick and short post, let me show you an example of it in this article here um, so typically in this short post, you won’t have any detailed explanations of the scientific processes that they are addressed and that will make this short.

This short post, very quick and easy to read, which means that they will typically target a quite general audience and also this short post can be very photographic, uh, for example, focusing on one central figure or picture as in our example. Here, another option is to write a long and more detailed post, as in our second example here, and the idea here is to tackle the subject more in depth and really take the time to explain the concepts and the scientific processes that are related to that subject, Which means that this longer pose can be a little a little bit more specific and maybe harder to follow if you’re, not from that specific field of study, which means that they might be addressing an audience which is a little bit more specialized.

You could also decide, as in our third example, to interview a person of interest and to then to transcribe the questions and answers of that interview in a nice way within an interview post, and the big advantage of these interview posts is that they will be very Personal and then easy to read and follow, and finally the last example that we are going to show you is opinion pieces such as the sassy scientists pose from the uh geodynamics division, blogs um.

So typically, the opinion pieces are are very subjective posts that leave room for humor or even sarcasm. If you want to so now, we propose you to pause the article and browse through the eg blocks for inspiration and try to choose a format that would work well. Both with your topic, but also the audience that you are aiming for so i’ve just shown you different formats that can be used for a blog post, but you might have noticed that actually um.

They are all based on the same, very basic principles that we call here the main pillars behind every good blog post and these main pillars are a good storyline, a catchy title, a hooking introduction, a fluent main body and nice pictures and layouts. So what we propose? You to do here is to grab a pen or maybe more uh, grab your keyboard if you prefer and actually start writing your own blog post.

So if you’ve never written any blog posts before or even if you have, but that you don’t feel very comfortable with it, don’t worry because we are going to guide you to guide you through each of these main pillars and give you some tips along the way And also note that we have chosen to organize these pillars in a specific order, which is, of course subjective and so feel free to um adapt it according to what feels right to you now before we get started.

Let me give you a first tip, which is don’t be afraid that your blog post is not perfect. Keep in mind that you are only making your first draft here and that the most important thing is to get started and then let the blog editors help you over the line. If you decide to submit your drive to our team after reading this short course so now, let’s start writing. So if you want your blog post to be fluent and easy to read, which is a very good thing for a blog post, a first important step is to try to define a good story for your blog post, meaning to decide what you are, how you are Going to organize your ideas, so try to outline what you are going to write in your post and a good tip for that could be to ask yourself some questions.

For example, what is the key message or information that you want to transmit to the reader, or why should the reader be interested in that specific topic or again? What is the information that is needed to transmit your key message in an efficient way or what are the implications for the reader or even for the future, so again, keep in mind that, according to the format or the audience that you’ve chosen for your post, you Might want to limit the number of questions that you are going to cover in your post and maybe focus only on one or two of these examples of questions here um so now.

The next step is to try to organize your answers to these questions. In a logical order and that logical order will make what’s what we call here, the structure of your blog post. So here to give you some inspiration, we go. We show you an example of a blog post with a very well thought structure, and so in that blog post, which is called the journey of a snowflake. The author wants to explain us the formation of snowflakes and so to do that.

She will start by explaining us what exactly is snow and then she will tell us the journey of a snowflake by going from waiter vapor to ice crystals and then from ice crystals to snowflakes and finally, from snowflakes to building a snowman. So a good tip that you could get also from reading these examples here is that you could use these questions or even the answers to these questions as subheadings to guide your reader through reading your posts, so now go ahead and try to list all the questions That you want to answer to in your post and organize bullet point them in a logical order to try to build the structure of your post.

So now, a next important step in writing and drafting your blog post is to write a catchy title. So you’re a headline is quite an import, important element of your blog post because it will really influence whether the reader wants to read the rest of your post or not. So a good headline could really make all the difference. So we know that you haven’t started. Writing your blog post yet, but we think that having a working title in mind, while writing your posts is a good thing because it can provide you with a sense of direction.

As you write and, of course, once you finish, drafting your blog post feel free to change and adapt your title into something that maybe provides a more accurate description of your final content. So now how to draft a catchy title so try to craft a headline that will accurately represent your post, but without sounding boring, um, also try not to be too long and maybe to limit yourself to six to eight words or around 140 characters.

So this is what is considered to be optimum for a good headline and finally – and maybe most importantly, try to be catchy and to do that, you can, for example, use puns or famous references, or you could even ask questions directly to the reader. For example, did you know that so here, to give you some inspiration, I will show you examples of headlines that were used within the ego division blocks.

So now what about you? So what could be a catchy title for your post and what i’ve proposed you to do now is to try a few headlines and show them, maybe to your friends and family, to try to figure out whether whether these headlines would make them want to read the Rest of your post or not, and next another very important element of a good blog post, is to having to have a good and hooking introduction.

So again we give you some tips here so try to hook people in with bold salesmen, so they want to read the rest of your blog posts. Also try to talk about what is going to be covered in the post, so to give the readers a quick overview of what they are going to learn, but don’t tell everything. So they still want to read more about your post and finally, and as valeria said before, try to use good keywords in your introduction again here we show you some examples of good introductions that were used in the egu blogs um to help you to get some Inspiration to now you draft a few sentences that will introduce the reader to your subject, but without revealing all of it and now for the following.

I will give the flow back to valeria all right, so you have maybe a catchy title by now. Maybe a short introduction: what about the rest of the post? So what you want to do now is create a good flow in the main body, so you may have convinced your reader to start reading, but now you have to keep their attention up until the end. So try to guide really your reader through your text by removing any possible obstacles.

Some of these obstacles may be scientific jargon, and we already talked about it before, but also try really to translate transition in a fluent way between paragraphs and don’t be too long and really try to avoid big walls of text. They are scary for readers, especially online. So try to use short paragraphs six to eight lines and not long sentences, usually 15 words work best and, very importantly, by the end of your blog post.

Deliver what you promise in your introduction, not something else, and you may deviate a little bit in their main body, but try not to go too far from your main subject. That could be the subject of another blog post later on that really try to stick to what you want to deliver in the introduction until the end. So an example of um connecting fluently paragraphs uh is from this blog post on the tectonic and structural science division block.

Let’s read it together so for centuries, chibita has been fighting against the natural degradation of the cliff, with recurring landslides slowly taking down the edges of the plateau, the top of the plateau consists of a 20 meter thick layer of consolidated rock formed from a can cache. Yeah, you might say that, well, I don’t really see why it doesn’t work well, it may work, but definitely the two paragraphs are not fluently connected one to the other.

If we add just a little more general introduction to the second paragraph, it reads much better. For centuries, chivita has been fighting against the natural degradation of the cliff, with recurring landslides slowly taking down the edges of the plateau, the geology of the plateau explains why this town is so prone to landslides. The top of the plateau consists of a 20 meter thick layer of consolidated rock formed from volcanic ash, but you can carry it much better.

I think so so. Try now to write individual paragraphs following your predefined stories with your introduction and try to make sure that there is a good reading flow until the end. And once you have done all that, you may go back and look at the layout of your blog post and make it pretty, because also the layout plays a big big role in convincing your readers to stay from start until the end, for example, keep in mind That out there there are quick readers, they want to pick keywords or key concepts and have them very visible in your main text so that they don’t have to read all of it if they don’t want to so make sure that your introduction stands out like we Said you can use sub eaters or in a different font, color or using golden italic, also in the middle of the text.

If you want to maybe underline a specific word of concept, please try to leave enough space enough white space between the different paragraphs and around images. These also help the eyes not to get too tired and use quotes to break the text so again, with maybe main concepts so that they are catchy and readers can can easily find them and another very important part of a blog post are images within it and That upcom accompany your text and sometimes gives the plus to the content of your post.

So try always to choose a header image and make sure that it stand out. That’s it’s also very important for later advertising. Your post, for example, on social media, but what’s really important and we like to stress out always credit the images you use, don’t steal them, check the copyright and of both photos and scientific figures and make sure that you are allowed to use them and again credit.

The owners there are, of course, sources of free images. For example, here we show some some website and in this presentation all the images came from unsplash or pixabay unless otherwise credited other sources of free images are, for example, national organization like nasa or isa and to wrap up on the layout and make it pretty um. Let’s go for two examples of the same blog post: in the first case, the left side, not much layouting was used.

So the text is all um the same size, font, the same same font, size uh. There is no bold or no italic. There is no quotes. Standing out so it’s pretty heavy. Also the white spaces are not that many. The paragraphs are really long and dense. Let’s see what happens when we add some subweddings or we use a little bit of a bold, to highlight the introduction. We also breaks break the paragraphs. A little bit more definitely this will read much more easier and won’t tire your reader eyes.

So now you have a text ready. Maybe a draft and the content is great, but what about the layout? So, let’s make it attractive. So now, head images figures: try to use different fonts play a little bit around with spaces and and white spaces as well, and make the structure as clear and as attractive possible, and now we pass the ball again to clara for the last part. So now we’re approaching the end.

Ideally, your blog post is ready, finished, looks nice and it is time to publish you have already decided before where you want to publish. So you publish it through your preferred outlet. Hopefully, you have gotten a bit of feedback before so that you are happy with what you publish and now comes the time that you want it to be read. So it’s time to share it. You can share it with your friends, your colleagues, your family, uh with other people as well, but of course you want to reach a larger audience, because you spend a lot of time.

Writing this blog post and you think it um. It is interesting what it contains. So you can reach also your target audience via email. If um, you know some lists, for example, who where people would be interested, but also on social media. Each social media has a different way of highlighting content, so keep in mind how you want to present it, but what they all have in common is that, additionally, to a catchy title, you want to write a short teaser of one to two sentences to get the People to click on the link to the post.

To do so, you can hook the reader with an analogy or a connection to current events or emotions you have to. As the word hook says, you have to get the reader with your teaser and your title so that the reader will click on the link and read what you have spent so much time and effort on. You can also use appealing keywords, as valeria already mentioned before. Hashtags are used in several social media platforms and again here either a header picture or if you, for any reason, did not use a header picture.

Another picture to increase the visibility, always remember what you would find interesting to click on, because this is also what other people might be interested in and as a closing exercise think about one platform. You would like to share your post on think about how what content this platform is specialized on write a short teaser to accompany your link and find a fitting picture if you’re not happy with your header picture, for example, so that’s it.

We hope that you have learned a lot. We had a lot of fun preparing this short course and we hope to hear your feedback or to read your drafts. We hope you enjoyed it, do not hesitate to contact us if you have any question if you’re interested in joining a blog team, if you do not know where to start, but you really like to do some science blogging do not hesitate to contact us. The worst is that we don’t answer you, but I don’t think we’re there.

We are motivated. We want you to communicate your science. If you have done all the practical exercises. Send us your draft and we’ll try to give you feedback in the next weeks, depending on how many drafts we get. We can also help you to get it published on one of the etu division blogs if you’re interested, if you don’t have your personal blog and you can find the email to which to send it here.

This will reach all of us here again if you’re interested in the links we had in the presentation you can find them in the youtube description below or in the slides, were also linked below. Thank you very much and yeah, and looking forward to your drafts, you

---

Don't have time to do the blogging thing?

Maybe Copywriting services are for you.

 

 

I know it is Saturday and it Saturday evening, and I believe that you would love to be anywhere else instead of being here thinking, but for now I would like to share a bit on the enforcement learning for marketing blog. Optimization touch me. This will be interesting, but then, for those good things that way there is a marketing there. Isn’t it too commercial? Let me clarify it this way.

This use case I’m using it to make sure that the case that we are using in the enforcement learning can tackle the real business problem and the way we solve this problem is quite broad so that you can apply this to many other problem related to your Industry and by the end of this presentation I will mention a bit on how enforce the other industries. So, let’s start in the next four thirty minutes, I think or 45 minutes.

Oh, we will discuss this topic in the beginning. We will start to discuss on the background while we discuss on marketing general problem and then we start to discuss on the enforcement learning about the h and we will combine them all and check the result. And finally, I will give you a bit on the final remark, so why marketing blog problem? I know this sounds a bit commercial. But if you are it at the article that tiny, if you cure that at the five four seven it has a lot of zero behind it.

So it’s no longer a trivial problem. We need to be quite wise to manage that budget allocation and not because it’s about money, but because it’s related on how our company will grow in the future and it’s in the same time. We want to make sure that the money is used well, so that the financial stability of the company, but I would like to share a bit on how it works. Basically, our marketers in Africa will define a particular level of budget and they will allocate the budget for particular marketing blog and from that marketing.

Blog user will see the ads and that will be counted as impressions and then, after the user clicks City ads. Some of them will be able to click the ads and those will be counted as clicks and after user clicks. Yet a fraction of them will be encouraged to buy the hour flight ticket or hotel ticket, and those will be called as customer acquisition and from customer acquisition. We cut our money if we see the problem earlier.

We know that there are two different objectives and it’s completely different in one side. In this side you see that marketers. If they’re KPI is solely on increasing customer acquisition, they will be very encouraged to pour out as money too much increase customer acquisition. But there is not a department or finance department and they won’t like it. We want to make sure that the budget that you pay to marketing blog agent is not infinite.

We want to make sure that it’s within our budget, so these two departments will keep debating maybe all day long, but maybe they keep it like one hour but in general, there’s two objectives are quite in the opposite side of our company. So how to approach this problem, how to make sure that the budget that we give to marketing blog action will be efficient well. In the same time, we can also manage to increase our customer acquisition.

Let’s take a step back on how to approach this problem by taking another analogy, because my patron is in supply chain and understanding. Let’s take a analogy from manufacturer, let’s say that you have a manufacturing and you just buy this robotic arm and your robotic arm. You want to make sure that this robotic arm can pick up ox for you, because paying people now is more expensive. We cannot put implementing it in two different perspectives.

That part is from optimization perspective, and this pink part will be in reinforcement, learning perspective. In optimization perspective, its conventional, you just make sure that the mistakes in picking how we will toss off the action taken in time T given that tip will be from 1 to n. Let’s imagine that taro Patek I’m in the position, zero and then PI in time. 1, the robotic arm will move up, it may be 15 decrease and maybe in time to the robotic arm will move, like maybe 30 degrees until time 10, for example, where that robotic arm is able to pick up the box.

But that requires planning and planning requires optimization. So, let’s imagine if we shift the box up it, then we will need to rerun the optimization algorithm again to make sure that the action taken is is able to fulfill our requirement. Let’s move to the right path, it sounds easy in reinforcement, learning which has stitched the robotic arm, pick the box, but then, if the robotic um take the wrong side, for example, let’s say the box in the left side and then the robotic arm moving for moving To the right side – and you say that the robotic arm it’s wrong – you should move a bit to the left, so that’s the basic of how the informal learning learning can be implemented in our analogy now, as we understand this analogy, the sting of the robotic arm As the technician that is required for the marketers to take in daily basis in terms of budget allocation in day one, they need to allocate particular budget and, in day two.

They also need to allocate particular budget until the end of the month. For example, where they’d want to make sure that the whole KPI, for example, the efficiency of their budget, is optimized, so by understanding that limitation, we finally chose to use reinforcement learning. Why? If we look at this table, there are three features equation to develop: computational efficiency and new trata. The first two, I think.

Maybe we are kind of understanding that optimization and reinforcement learning is on the same crown. It’s like I can take either optimization and reinforcement. Learning. No matter, whichever method that I will take because in optimization I will only need to think about objective and constrain, but in the enforcement I still do think as well, but I just need to define the word function: national efficiency in optimization.

It depends on how you formulate a problem. If the optimization problem is in terms of linear and it’s solvable, then it just takes maybe one or two second to solve the problem. But if the problem is nonlinear, then it will take some time to explore the solution. Space and it’s not good, especially when you’re marketers is very demanding and then, if you are using reinforcement, learning the training itself texts quite and if or even for simple problem.

But then, what the main reason of for us to use reinforcement learning is because of the third feature that I mentioned. We are expecting that we will have new data in daily basis and if we are implementing optimization here, imagine that, if we implement optimization having new data in telophase implies that we will have a new solution space in daily basis as well, and it will require us To keep rerunning the algorithm until you find the optimal solution, and that’s quite tedious actually and that’s why we are proposing to use informal learning.

Because, in the beginning, the training that takes a lot of effort will pay well, because it allows us to learn the parameter. That can map between current state and the proper action that should be taken to maximize tell you were checking who is here from computer science? Okay, so you guys are smarter than me. We will talk a bit on the enforcement running. I think you are already learning reinforcement, learning in college, so if I say something wrong, please correct me because I know you are from counter science.

This is just my simple understanding on what is reinforcement. Learning there is an interest and pheromone HN will have a current state and in the current state, action will take an action. The action taken will be received in the environment here with tech environment as our marketing general problem and from that action and pheromone will give response in terms of reward and that reward will allow the agent to move forward to the new state.

So to understand the thermos further, I think this one is for you. We haven’t learned, reinforcement learning here, we have motor motor is used to describe the environment, and policy is to describe how agent takes action and there is also a failure function and, according to my understanding, there are a lot of way to define value function in reinforcement, Learning because it describes the word of an action, and I think this is important, because the way you define the reward will you find how to hm detects the action and when you want to make sure that your agent texts, a proper action, you will need to Define value function carefully and here reward refers to the feedback based on the current step future.

What will you find cumulative this country work that I make this contact reward path, a pitbull and if you see the blue, the blue part and make it smaller of it, because it’s just count and then there is also state value action value, an advantage. The last three, I think, is closely related. State value is basically expected expectation of future. You are given particular state and action value is the expectation of future reward given state and action, and the advantage is the difference between put that value and action value to make it more intuitive.

Let’s think that if you have a higher action value, it implies that the decision taken given a particular action is better than having the average having the average that value, because, basically, that value is the average of everything. But the action value is the average given particular state and action. Now we want to learn more about the agent, so we will discuss this left part. We know that agent takes action, but then how does the agent learn to take good actions? There are a lot of way for the agent to learn to take good actions.

But here what I want to explain is policy Creighton method in which, basically, the agent that part will have a particular parameter that will be used for the policy to take action and that action will be implemented in the environment and the environment will result, in particular Value function and from that particular fault function there will be an optimization process from which the value function will be used as the basis to optimize the parameter use in the policy, so that one is just my rough understanding to be more specific.

What we are using now is proxima policy optimization, it’s specifically a type of policy creation method, except the three different plot that I have colored with in there. So we have aging and it still has particular parameter and it still has fallen see we will purchase we’ll use the parameter to purchase action sampling instead of action itself. So if you imagine, because we are here facing a marketing budget problem, it means that our problem is in the continued space.

So taking a deterministic action decision will not work. Well, that’s why we need an action sampling and this action sampling will be implemented in the environment and here from the hole, 5 value function. We will use advantage to peel the passage for the optimization and we also implement sort of constrain to make sure that the update on the policy parameter be drastically different, because a drastic update in PPO means that the algorithm will perform very unstable.

So here I would like to emphasize why we choose PayPal, because there are four point that I mentioned. Basically, the first point: maximize reward based on the policy is the strength of PPO, because it’s a part of policy clarity and method. Basically, it allows PPO topper performance better in continuous space and the use of action simply lost better exploration and the use of advantage lost a reduce variance.

The fourth point, I think, the key of understanding people, because if you see the passes of PPO is the RPO, which is a translation policy organization, and I think translation optimisation is different from the usual condition. I think we are underst, so transaction optimization is a different family of optimization, usually in machine learning. We understand condition as a part of line search method.

Basically, it stops based on the Creighton but introspection optimization. You are stepping within a certain translation, so you do not want to a step further if it’s outside your translation, because, basically you don’t really trust outside the transaction boundaries, and this is implied in PP OPA using clip mechanism, and this will help the algorithm to perform In a stable wide way so that it can help us during the training period.

Now, let’s talk about the environment, the environment is in the dead part. The thing unreal environment is expensive and it’s left it’s less likely for your company to pay for your experiments. What we do here is to place this complicated part with a single motor. It’s a simplification. Basically, we try to imitate what happens in the real environment so that we can use that model to sort of simulate what happens in the real world problem, but how? How can we create our model? There are two different approach here.

Direct who have heard about regression can trace Juergen. Should I explain okay so for those who has an understand, I think you can cook a letter, but the question is basically the way for us to map between input variable to make sure that we can produce output variable. So in the regulation we are treating data as random sample and the parameter is assumed to be fixed and the difference between data and twelve output will be treated as error.

That’s right there. We are only seeing one line because all is assumed to be fixed and everything else is computed as error. But if we see that one particular line, we see that there is no uncertainty there, because everything else is assumed as error, and we cannot use here this regression into our enforcement algorithm. Because you see you want to make sure that your agent is able to detect as attack as possible.

But then, if you are teaching them with fixed input, then you will be sort of doing a very non productive things. So what we do here, we are looking at patient recreation. Have anyone heard about patient attrition? No one, so patient regression is basically, I think, another way for us to approach more telling, instead of treating parameter as freaks, we treat parameter as probabilistic. In the second point, we will need Pryor for that, but it’s worth the effort because, basically, if you see there are a lot of lines because it’s trying to capturing the potential of uncertainty and that’s what’s happened in our real world and that’s why we are using Patient, so in the environment, we are using position regression.

Why? Because, even if the training takes effort, we are able to capture uncertainty and that allows us to teach the agent to text more smarter decision and we will combine them all so far combined. We have learned about reinforcement, learning agent, we have learned about reinforcement, learning environment and we have also learned about patient education, but then how to put things into places here.

I have created a sort of very simple map on how to combine those different things into one reinforcement: learning that allows us to take smarter decision. Oh, we will not learning it in sequence, but we will start with the more telling first, the first point, and then we start to learn about the beginning of the algorithm, which is in the second part, and then we will learn. The third part, which is the final part of the algorithm, so we will start in the first point, remember that in the mottling we are using patient regression in position regression, instead of assuming that y equal to a times X, plus B, we are assuming that Y.

Following particular distribution and that’s what happened here not say that Y is impressions and the X is action and holiday. We are treating impression as a sort of random variable that follows negative binomial distribution with parameter mu I and theta, and here parameter mu. I is defined as exponential of the long things insert a bracket, but if you see that long part inside the bracket is sort of similar to linear regression correct, so it’s like one step forward going from the usual inner equation that is not April to capture uncertainty.

To another met method called patient equation that can capture uncertainty. The same goes with link clicks. Basically, we assume that link clicks also following another tissue portion of negative binomial, with particular parameter that following the equation equation inside the brackets and in the end we have free worth, which is defined as by justified, appealing clicks and we have a lock function there and Minus sign there and this will be an input for componentry.

If you remember, abhorrently is the last part of the algorithm question: how to implement position regression Wi-Fi Stan because it has an extensive documentation, it it’s quite reliable and it has a variety of functions and for those who have read Python documentation, it’s quite extensive and it Is able to capture Safra function that maybe is not available in utter patient, patient type for off library such as pi, MC or edwards? Next, in the second part, this part, I try to simplify it to make sure that it’s similar to the one in policy creation, but in our implementation we are modifying a bit to make sure that there is also a part of it that can help the agent To also predict well the reward that will be given by the environment, so, instead of having policy network only, we will have also value Network policy network, as its name is a network that is used to sort of taking a good position, and this can be done As policy Network producing action parameter and that action parameter is used for action, sampling which will go to the first part of the our map, that computer part and the failure network will be used to predict state value.

And this will go to the third part of our map, so value network and policy network. The input is only state and if you want to make it more complicated, you can also take into account your action into the network, but that will be in other cases and then this is the and a combine our reinforcement, learning algorithm. We will calculate the advantage and also make sure that it will be optimized. How remember that, for this third part, we have two different inputs, pre-work from the first more telling in python, and also predicted state value, which is come from the second part of the map.

And we can use those two to make sure that we have an optimized parameter for our value network and in the same time, we can also use that to calculate the advantage and then make some adjustment by using importance, weight and prep mechanism. And we can use it to optimize policy network so that we can improve the parameter for policy network, but seems the computer requires we can okay for the algorithm.

We use my touch here. Why? Because this is still our initial study, so we are planning to develop it mark. But for now, because this is initial study, we will require a lot of test and error to check what happens in the when it’s exposed to real environment. But in short, we are using five touch because it’s flexibility and it’s also easy to debug and also for convenience, especially for pyro. When you are understanding that you want to make sure that there is violation mechanism that you want to involve in your algorithm.

Having pyro. In your library is quite and convenient because it allows us to explore probabilistic rocker programming in our neural network, so Theresa. If you see here after a 100 iteration, the failures quite converse and importantly, the reward is also keeps increasing until it saturate at around 8800 iteration and from there. I would like to continue to finally map what we have done so far. We have made sure that we see it a real-world problem and we use that real-world problem to Motel the agent temperament to make sure that we have equipped erm water and then we are training and checking the result of the motel, which that is kappa /.

A simple rational decision maker, but then what’s next in term of motor component, especially in the past, an area. Personally, it’s my interest to adding to add dependence to the other states and maybe also to add seasonality Antron, because this initial study, we try to make it as simple as possible to make sure that it’s really rational, as it says, and in term of other project. This is for you who thinks that marketing is too commercial, but by understanding all of the framework that I have explained earlier.

I think, having this framework to be implemented in production planning, logistic and industrial automation is also worth to try, especially because, if you think further, because in production, planning and logistic, usually the one who implemented those three. Those production, planning and logistics are human and most of the time they are quite flexible and adaptable to see what happens in the real environment, but in the same time they also have pauses, and this process and their colleagues sometimes make some point, be rational and I Think it’s worth to try to see what happens when we are developing a sort of smarter agent to make an automatic decision-making so that we can at least guide the human who texted realization to make sure that decision is still rational along the way.

So, for reference, this is based on the compilation of the work of many people, especially from the pepper from Sherman, and also a lot of tutorial from many other article and for those who wants to learn more about passion. Statistic I’m suggesting the tenth point, and this is also the result of me asking so many people and I’m glad that they are not kicking me out yet and also the marketing team that allows me to explore this use case for PyCon.

Thank you measure, Koscheck. Okay, with a kind of no it’s a country – Oh so let’s say that you have a solution: space tip to speak, but in PPO you try to limit yourself. Also, let me start it again. You have this ocean space to speak and you are in this point and the optimization algorithm says: if you want your parameter to be Aquaman, you have to jump that far, but in PPO you do not want to jump that far, because jumping that far will allow Your algorithm to be unstable, so you are limiting yourself in a tiny Q peak to make sure that you are not jumping that work, but something this small and that constrain is implemented in a simplified way by using creep mechanism with mechanism is basically just saying.

I just allow myself to start as much as maybe s 1 centimeters instead of 9 kilometers, the bundle. Oh so it’s a hyper tuning parameter in PP. Oh, it’s like an epsilon parameter where you allow 1 plus epsilon and 1 minus Epsilon. So this is the way the data is presented. Basically, we have a temple, the content, historical data and that historical data basically have an X in terms of action and quality.

Whether the action is taken is in the holiday or not, and we use it to train how many impressions and how many links and how many reward, but this model is used to make an imitation for the real environment. So it’s used as the black box here. If you have freed a bit on how to implement the enforcement learning most of the time, we are not using sort of more telling all most of the time they use important AI.

You try to run a simple panda loom or to make sure that a car will move around, but that sort of physical mechanism is already defined by the library team ai. But here we are replacing that library by using patient relation very good question. My god is quite messy right now and I think, because of the limitation of time I haven’t, submit my cut to the HR. In the same time, I need to make sure that my business trip is reimbursed.

So I think after this you can email me and maybe we I can make sure that the code is quite neat for to be shared yeah. Oh no! No, the one that we compare is the basic one, because when we replace this with when very, but when we are intending to replace human with this edge – and we want, they will be very afraid correct. So what we are comparing is what the real human will take, the decision it will take and we compare it with the result of this algorithm and the difference is not that big, I’m not expecting for the algorithm to be in the most Cockerham.

But the difference is much less than 5 %, so I think for the potential of engine optimisation for the for the potential of having make having to make it automated. I think it’s worth in the future to tie my father. Excuse me: can you motel free? Oh, I think this one is more Terry yeah, PPOs, motel free so for a particular marketing blog. We will have time for marketing teams and they are sort of giving us the collection of historical data or in day one how much money they pay and how much impression ticket and link clicks.

Take it it’s sort of that data horses, Oh

---

Click here to get 2000 4K stock videos today!