cs209搜索广告平台第一周lesson

A B C杜聿明 D陈数据结数据结SearchAds(搜索NativeAds(原生DisyAds(显示SearchAds(搜索logic:match touser’sadsformat:text,adsposition:mainline,sidebar,top,bottomofsearch NativeAds(原生logic:match towebpageorAPP’sadsformat:text,image,styleshouldalsomatchcontextofwebpageorAPPadsposition:embeddedinoriginalcontentofwebpageorDisyAds(显示 logic:matchuser’sdemographic,intereststoad’scategoryinterestscollectedfromuserbehavior:pagedwelltime,click,engagementadsformat:image,animation(gif)，,adsposition:sidebar,top,bottomofpageoryou’re475timesmorelikelytosurviveanecrashthanclickabanner互互联What’sinCommonlogic:matchAdstosomething:user’squery,interests,contextofwebpageor互互联Quiz:Whichonehashigherclickratenativeads>searchads>y数据结数据数据结数据数据结n●●●●Anfocusesonathemeoragroupofsetachooseyouraudiencewriteyourad,ad数据结数据结数据结given603click,16Kimpression,what’stheclick$300budgetdaily,$1.4perclickwhydailypotentialclicksis

数据结

数据结Queryremovestopwords:a,an,theremoveendings,queryintentpredictuser’sbuyharrypotterdvdonline->harrypotterneedqueryhistoryquerynikerunningshoe->nikerunningsoftwaredeveloper->softwareengineerresult:alistofquerySelectsendqueryunderstandresulttoindexandselectasmuchcandidatesasapplyinformationretrievalalgorithmonindexhowtomakeitscalablewhatifQPS>10K Rankgivenadcandidates,howtocalculaterankagoodrankscoreshould relevancebetweenqueryand click bid howtochargeisitfairtochargethembybidcostperclickincreaseisgoodnot 数据结Webcrawlingisaboutmakingrequestsandextractingdatafromtheresponsestep1httpstep2step3extractdatafromHTMLParseHTML<style><script><div><div>搜索平台实attributename:attributevalue 搜索平台实搜索平台实战搜索平台实WheretostartNetworkI/OistheAvoidBotCrawlerneededtobeWheretostartNetworkI/OistheAvoidBotCrawlerneededtobeWheretostartcrawlingstartwithfeedsfilealistofwebsiteurlrequesturlwithdifferentQueamplencornerleatherwhiteleather web

crawledNetworkI/Oistheifasitetakes2-3secondstorespond,thenyou’relookingatmaking20-30requestsaminutewasteCPUwhenwaitingforresponsefrommakeyourcrawlermulti-AvoidBotWhatifwebsite blockyourIPQuiz:Whatstatus5035xxServer503ServiceSpoofHttpHeaders:UserAgent,Accept,Accept-Service:RotateIPsusingalistofover100+CrawlerneededtobeWhatifcrawlercrashduetouncaughtHowtohandleurlfailedtocrawlHowtoavoidcrawlsameurllogcrawledlogfailedlog数据结DeviceDeviceSessionAdQuery配置开发环Java，Python,JavaIDE(InlijorEclipse),MemCached(optional)，MySQL(optional),Spark(optional)参见预习资准备feeds文件，发sofa,12.4,deadline:12/31/20164PMPacific课程项目邮：

，获得太阁信 搜 ●●●配置开发环Java，Python,JavaIDE(InlijorJavalib:Jsoup1.10.1，jacksoncrawldedupegenerateclicklogforqueryintentpredictionandclickreverseDeviceDeviceSessionAdQueryHowtodevelopfacerecognitionalgorithmwithfacewhatifwedon’thavefaceimageatWealsoneednegativePositiveQueryCategry_AdsCategoryQuery_nIdQuery_AdIdNegativemismatchedquery_categorymismatched mismatched nIdweight,lowestadIdweightperqueryStep1:Segmentusersto410000IP,5deviceidperIP,user=level0:5%ofuserwhoclickforeachlevel1:5%ofIPwhose1st2deviceidclickforeachquery,70%ofquery,restof30%querynoclick

rest3device_idclicklevel2:40%ofIPfromwherewerandomselect1device_idclickfor50%ofquery,restof50%querynoclick,restof4device_idnoclicklevel3:50%neverStep2:assign nID,AdId,Click(0/1),Adcategory_Querycategory(0/1)touserassignweight nID,AdgroupaddatabyQuerygroupngroup1corner1leather1whiteleather1officecomputeradjustableofficehomeofficemetalofficenAdHowtocalculateweightforeach Foreachad n,calculaterelevancebetweenadandquery,=relevance_i/sum(relevance_i),iisindexofadfor Foreach nperquerygroup, n_weight_j=sumofadrelevancein n/total_relevance_cross_ ngroupId1corner1leather1whiteleather1officecomputeradjustableofficehomeofficemetalofficenAdHowtoassign nID,AdIdtoWeightedRandomGivennitem,eachonehasweightw[i],theseweightsformtheunnormalizedprobabilitydistributionwewanttosamplefrom,each