开源网站流量统计系统Piwik源码分析-参数统计（一）

1、 Piwik现已改名为，这是套国外著名的开源站统计系统，类似于百度统计、Google Analytics等系统。最的区别就是可以看到其中的源码，这正合我意。因为我直对统计的系统很好奇，很想知道的运原理是怎么样的，碰巧了解到有这么个系统，因此马上尝试了下。国内关于该系统的相关资料较匮乏，多是分享怎么安装的，并没有找到有关源码分析的章。下先对其做个初步的分析，后会越来越详细，本前的职位是前端，因此会先分析脚本代码，后再分析后台代码。、整体概况Piwik的官是，使PHP编写的，我以前就是PHP程师，因此看代码不会有障碍。前最新版本是3.6，Github地址是，打开地址将会看到下图中的内容（只截取了关

2、键部分）。打开js件夹，的piwik.js就是本次要分析的脚本代码（如下图红框出部分），内容较多，有7838代码。先把系统的代码都下载下来，然后在本地配置虚拟录，再开始安装。在安装的时候可以选择语，该系统持简体中（注意下图中红框出的部分）。系统会执些操作（注意看下图左边部分），包括检查当前环境能否安装、建数据库等，按照提步步来就，较简单，没啥难度。安装完后就会动跳转到后台界（如下图所），有图表，有分析，和常的统计系统差不多。功能还没细看，只做了初步的了解，界的友好度还是蛮不错的。嵌到页中的JavaScript代码与其它统计系统也类似，如下所，也是异步加载的式，只是发送的请求地址没有伪装成图像地

3、址（注意看标红的那句代码）。/* tracker methods like setCustomDimension should be called before trackPageView */varu=/;/定义_paq.push(setTrackerUrl, u+piwik.php);_paq.push(setSiteId, 1);)();在页中嵌这段脚本后，页在刷新的时候，会有下图中的请求。在请求中带了堆的参数，在后的内容中会对每个参数做释义。7000多的脚本，当然不能的读，需要先拆分，拆成个个的模块，然后再逐个分析。脚本之所以这么，是因为编写了量代码来兼容各个版本的浏览器，这其中甚包括

4、IE4、Firefox1.0、Netscape等灰级的浏览器。接下来我把源码拆分成6个部分，分别是json、private、query、content-overlay、和piwik，如下图红线框出的所，piwik-all中包含了全部代码，便于对。代码已上传到。json.js是个开源插件，为了兼容不持JSON对象的浏览器设计的，这的代码可以单独研究。private.js包含了些于全局的私有变量和私有函数，例如定义系统对象的别名、判断类型等。query.js中包含了很多操作HTML元素的法，例如设置元素属性、查询某个CSS类的元素等，它类似于个微型的jQuery库，不过有许多独特的功能。conte

5、nt-overlay.js有两部分组成，部分包含内容追踪以及URL拼接等功能，另部分是来处理嵌套的页，这具体没有细看。tracker.js中只有个Tracker()函数，不过内容最多，有4700多，主要的统计逻辑都在这了。piwik.js中内容不多，包含些初始化和插件的钩等功能，钩具体怎么运作的还没细看。虽然分成了6部分，但是各部分的内容还是蛮多的，并且内容之间是有联系的，因此短时间的话，很难搞清楚其中所有的门道。我就挑了点我个感觉最重要的先做分析。1）3种传送数据的式我原先只知道两种传送数据的式，种是通过Ajax的式，另种是创建个Image对象，然后为其定义src属性，数据作为URL的参数传

6、递给后台，这种式很通，并且还能完美解决跨域问题。我以前编写的个性能参数搜集的插件，也是这么传送数据的。在阅读源码的时候，发现了第三种传送数据的式，使Navigator对象的。MDN上说：此法可于通过HTTP将少量数据异步传输到Web服务器。虽然这个法有兼容问题，但我还是被震撼到了。它很适合统计的场景，MDN上讲到：统计代码会在页关闭（）之前向web服务器发送数据，但过早的发送数据可能错过收集数据的机会。然，要保证在页关闭期间发送数据直较困难，因为浏览器通常会忽略在卸载事件中产的异步请求。在使sendBeacon()法后，能使浏览器在有机会时异步地向服务器发送数据，同时不会延迟页的卸载或影响下页

7、的载。这就解决了提交分析数据时的所有的问题：使它可靠，异步并且不会影响下页的加载，并且代码更简单。下是代码段（注意看标红的那句代码），存在于tracker.js中。function sendPostRequestViaSendBeacon(request) varsupportsSendBeacon =object = typeof navigatorAlias &function = typeof navigatorAlias.sendBeacon &function = typeof Blob;if (!supportsSendBeacon) returnfalse;varheaders

8、= type: application/x-www-form-urlencoded; charset=UTF-8;varsuccess = false;try varblob = newBlob(request, headers);success = navigatorAlias.sendBeacon(configTrackerUrl, blob);/ returns true if the user agent is able to successfully queue the data for transfer,/ Otherwise it returns false and we nee

9、d to try the regular way catch (e) returnfalse;returnsuccess;2）参数释义下的法（存在于tracker.js中）专门于搜集页中的统计数据，将它们拼接成指定链接的参数，这条链接中的参数最终将会发送给服务器。/* Returns the URL to call piwik.php,* with the standard parameters (plugins, resolution, url, referrer, etc.).* Sends the pageview and browser settings with every requ

10、est in case of race conditions.*/function getRequest(request, customData, pluginMethod, currentEcommerceOrderTs) vari,now = newDate(),nowTs = Math.round(now.getTime() / 1000),referralTs,referralUrl,referralUrlMaxLength = 1024,currentReferrerHostName,originalReferrerHostName,customVariablesCopy = cus

11、tomVariables,cookieSessionName = getCookieName(ses),cookieReferrerName = getCookieName(ref),cookieCustomVariablesName = getCookieName(cvar),cookieSessionValue = getCookie(cookieSessionName),attributionCookie = loadReferrerAttributionCookie(),currentUrl = configCustomUrl | locationHrefAlias,campaignN

12、ameDetected,campaignKeywordDetected;if (configCookiesDisabled) deleteCookies();if (configDoNotTrack) return;varcookieVisitorIdValues = getValuesFromVisitorIdCookie();if (!isDefined(currentEcommerceOrderTs) currentEcommerceOrderTs = ;/ send charset if document charset is not utf-8. sometimes encoding

13、/ of urls will be the same as this and not utf-8, which will cause problems/ do not send charset if it is utf8 since its assumed by default in PiwikvarcharSet = documentAlias.characterSet | documentAlias.charset;if (!charSet | charSet.toLowerCase() = utf-8) charSet = null;campaignNameDetected = attr

14、ibutionCookie0;campaignKeywordDetected = attributionCookie1;referralTs = attributionCookie2;referralUrl = attributionCookie3;if (!cookieSessionValue) / cookie ses was not found: we consider this the start of a session/ here we make sure that if ses cookie is deleted few times within the visit/ and s

15、o this code path is triggered many times for one visit,/ we only increase visitCount once per Visit window (default 30min)varvisitDuration = configSessionCookieTimeout / 1000;if (!cookieVisitorIdValues.lastVisitTs |nowTs - cookieVisitorIdValues.lastVisitTs visitDuration) cookieVisitorIdValues.visitC

16、ount+;cookieVisitorIdValues.lastVisitTs = cookieVisitorIdValues.currentVisitTs;/ Detect the campaign information from the current URL/ Only if campaign wasnt previously set/ Orif it was set but we mustattribute to the mostrecent one/ Note: we are working on the currentUrl before purify() since we ca

17、n parse the campaign parameters in the hash tagif (!configConversionAttributionFirstReferrer |!campaignNameDetected.length) for(i in configCampaignNameParameters) if (Ototype.hasOwnProperty.call(configCampaignNameParameters, i) campaignNameDetected = getUrlParameter(currentUrl,configCampaignNamePara

18、metersi);if (campaignNameDetected.length) break;for(i in configCampaignKeywordParameters) if (Ototype.hasOwnProperty.call(configCampaignKeywordParameters,i) campaignKeywordDetected = getUrlParameter(currentUrl,configCampaignKeywordParametersi);if (campaignKeywordDetected.length) break;/ Store the re

19、ferrer URL and time in the cookie;/ referral URL depends on the first or last referrer attributioncurrentReferrerHostName = getHostName(configReferrerUrl);originalReferrerHostName = referralUrl.length? getHostName(referralUrl): ;if (currentReferrerHostName.length & / there is a referrer!isSiteHostNa

20、me(currentReferrerHostName) & / domain is not the current domain(!configConversionAttributionFirstReferrer | / attribute to last known referrer!originalReferrerHostName.length | / previously emptyisSiteHostName(originalReferrerHostName) / previously set but in current domainreferralUrl = configRefer

21、rerUrl;/ Set the referral cookie if we have either a Referrer URL,or detected a Campaign (or both)if (referralUrl.length | campaignNameDetected.length) referralTs = nowTs;attributionCookie = campaignNameDetected,campaignKeywordDetected,referralTs,purify(referralUrl.slice(0, referralUrlMaxLength);set

22、Cookie(cookieReferrerName,JSON_PIWIK.stringify(attributionCookie),configReferralCookieTimeout,configCookiePath,configCookieDomain);/ build out the rest of the requestrequest +=&idsite= +configTrackerSiteId +&rec=1 +&r= +String(Math.random().slice(2, 8) + / keep the string to a minimum&h= +now.getHou

23、rs() +&m= +now.getMinutes() +&s= +now.getSeconds() +&url= +encodeWrapper(purify(currentUrl) +(configReferrerUrl.length? &urlref= + encodeWrapper(purify(configReferrerUrl): ) +(configUserId & configUserId.length? &uid= + encodeWrapper(configUserId): ) +&_id= +cookieVisitorIdValues.uuid +&_idts= +cook

24、ieVisitorIdValues.createTs +&_idvc= +cookieVisitorIdValues.visitCount +&_idn= +cookieVisitorIdValues.newVisitor + / currently unused(campaignNameDetected.length? &_rcn= + encodeWrapper(campaignNameDetected): ) +(campaignKeywordDetected.length? &_rck= + encodeWrapper(campaignKeywordDetected): ) +&_re

25、fts= +referralTs +&_viewts= +cookieVisitorIdValues.lastVisitTs +(String(cookieVisitorIdValues.lastEcommerceOrderTs).length? &_ects= + cookieVisitorIdValues.lastEcommerceOrderTs: ) +(String(referralUrl).length? &_ref= +encodeWrapper(purify(referralUrl.slice(0, referralUrlMaxLength): ) +(charSet ? &cs

26、= + encodeWrapper(charSet) : ) +&send_image=0;/ browser featuresfor(i in browserFeatures) if (Ototype.hasOwnProperty.call(browserFeatures, i) request += & + i + = + browserFeaturesi;varcustomDimensionIdsAlreadyHandled = ;if (customData) for(i in customData) if (Ototype.hasOwnProperty.call(customData

27、, i) &/dimensiond+$/.test(i) varindex = i.replace(dimension, );customDimensionIdsAlreadyHandled.push(parseInt(index, 10);customDimensionIdsAlreadyHandled.push(String(index);request += & + i + = + customDatai;deletecustomDatai;if (customData & isObjectEmpty(customData) customData = null;/ we deleted

28、all keys from custom data/ custom dimensionsfor(i in customDimensions) if (Ototype.hasOwnProperty.call(customDimensions, i) varisNotSetYet =-1 = indexOfArray(customDimensionIdsAlreadyHandled, i);if (isNotSetYet) request += &dimension + i + = + customDimensionsi;/ custom dataif (customData) request +

29、= &data= + encodeWrapper(JSON_PIWIK.stringify(customData); else if (configCustomData) request += &data= + encodeWrapper(JSON_PIWIK.stringify(configCustomData);/ Custom Variables, scope pagefunction appendCustomVariablesToRequest(customVariables, parameterName) varcustomVariablesStringified = JSON_PI

30、WIK.stringify(customVariables);if (customVariablesStringified.length 2) return(& + parameterName + = + encodeWrapper(customVariablesStringified);return;varsortedCustomVarPage = sortObjectByKeys(customVariablesPage);varsortedCustomVarEvent = sortObjectByKeys(customVariablesEvent);request += appendCus

31、tomVariablesToRequest(sortedCustomVarPage, cvar);request += appendCustomVariablesToRequest(sortedCustomVarEvent, e_cvar);/ Custom Variables, scope visitif (customVariables) request += appendCustomVariablesToRequest(customVariables, _cvar);/ Dont save deleted custom variables in the cookiefor(i in cu

32、stomVariablesCopy) if (Ototype.hasOwnProperty.call(customVariablesCopy, i) if (customVariablesi0 = | customVariablesi1 = ) deletecustomVariablesi;if (configStoreCustomVariablesInCookie) setCookie(cookieCustomVariablesName,JSON_PIWIK.stringify(customVariables),configSessionCookieTimeout,configCookieP

33、ath,configCookieDomain);/ performance trackingif (configPerformanceTrackingEnabled) if (configPerformanceGenerationTime) request += >_ms= + configPerformanceGenerationTime; else if (performanceAlias &performanceAlias.timing &performanceAlias.timing.requestStart &performanceAlias.timing.responseEnd

34、) request +=>_ms= +(performanceAlias.timing.responseEnd -performanceAlias.timing.requestStart);if (configIdPageView) request += &pv_id= + configIdPageView;/ update cookiescookieVisitorIdValues.lastEcommerceOrderTs =isDefined(currentEcommerceOrderTs) & String(currentEcommerceOrderTs).length? curren

35、tEcommerceOrderTs: cookieVisitorIdValues.lastEcommerceOrderTs;setVisitorIdCookie(cookieVisitorIdValues);setSessionCookie();/ tracker plugin hookrequest += executePluginMethod(pluginMethod, tracker: trackerInstance,request: request);if (configAppendToTrackingUrl.length) request += & + configAppendToT

36、rackingUrl;if (isFunction(configCustomRequestContentProcessing) request = configCustomRequestContentProcessing(request);returnrequest;统计代码每次都会传送数据，每次请求都会带上串的参数，这些参数都是简写，下做个简单说明（如有不正确的地，欢迎指正），部分参数还没作出合适的解释，例如UUID的成规则等。先将这些参数分为两部分，第部分如下所列：1、idsite：站ID2、rec：1（写死）3、：随机码4、h：当前时5、：当前分钟6、s：当前秒数7、url：当前纯净地址

37、，只留域名和协议8、_id：UUID9、_idts：访问的时间戳10、_idvc：访问数11、_idn：新访客（前尚未使）12、：访问来源的时间戳13、_viewts：上次访问的时间戳14、cs：当前页的字符编码15、send_image：是否图像请求式传输数据16、gt_ms：内容加载消耗的时间（响应结束时间减去请求开始时间）17、pv_id：唯性标识再列出第部分，于统计浏览器的功能，通过Navigator对象的属性（mimeTypes、javaEnabled等）和Screen对象的属性（width与height）获得。1、pdf：是否持pdf件类型2、：是否持QuickTime Playe

38、r播放器3、realp：是否持RealPlayer播放器4、wma：是否持MPlayer播放器5、dir：是否持Macromedia Director6、fla：是否持Adobe FlashPlayer7、java：是否激活了Java8、gears：是否安装了Google Gears9、ag：是否安装了Microsoft Silverlight10、cookie：是否启了Cookie11、res：屏幕的宽和（未正确计算清显器）上这11个参数的获取代码，可以参考下这个法（同样存在于tracker.js中），注意看代码中的pluginMap变量（已标红），它保存了多个类型，来检测是否安装或启了指定的插件或功能。/* Browser features (plugins, resolution, cookies)*/function detectBrowserFeatures() vari,mimeType,pluginMap= / document typespdf: application/pdf,/ media playersqt: video/quicktime,realp: audio/x-pn-realaudio-plugin,wma: