{"id":11083,"date":"2022-11-08T14:32:15","date_gmt":"2022-11-08T06:32:15","guid":{"rendered":"https:\/\/www.lwops.cn\/?p=11083"},"modified":"2022-11-08T14:33:56","modified_gmt":"2022-11-08T06:33:56","slug":"prometheus%e6%8a%80%e6%9c%af%e5%88%86%e4%ba%ab-prometheus%e8%87%aa%e5%ae%9a%e4%b9%89%e5%91%8a%e8%ad%a6%e8%a7%84%e5%88%99%e8%a7%a3%e6%9e%90%e5%92%8c%e9%85%8d%e7%bd%ae","status":"publish","type":"post","link":"https:\/\/www.lwops.cn\/index.php\/prometheus\/prometheus-artisan\/11083","title":{"rendered":"Prometheus\u6280\u672f\u5206\u4eab\u2014\u2014prometheus\u81ea\u5b9a\u4e49\u544a\u8b66\u89c4\u5219\u89e3\u6790\u548c\u914d\u7f6e"},"content":{"rendered":"\n
\u4e0a\u4e00\u671f\u4e50\u7ef4\u541b\u8ddf\u5927\u5bb6\u5df2\u7ecf\u4ecb\u7ecd\u4e86prometheus\u7684\u5b89\u88c5\u4e0e\u914d\u7f6e\uff0c\u5bf9\u4e8e\u8fd0\u7ef4\u76d1\u63a7<\/a>\u800c\u8a00\uff0c\u9664\u4e86\u76d1\u63a7\u5c55\u793a\u4ee5\u5916\uff0c\u53e6\u4e00\u4e2a\u91cd\u8981\u7684\u9700\u6c42\u65e0\u7591\u5c31\u662f\u544a\u8b66\u4e86\u3002\u826f\u597d\u7684\u544a\u8b66\u53ef\u4ee5\u5e2e\u52a9\u8fd0\u7ef4\u4eba\u5458\u53ca\u65f6\u7684\u53d1\u73b0\u95ee\u9898\uff0c\u5904\u7406\u95ee\u9898\u5e76\u9632\u8303\u4e8e\u672a\u7136\uff0c\u662f\u8fd0\u7ef4\u5de5\u4f5c\u4e2d\u4e0d\u53ef\u6216\u7f3a\u7684\u91cd\u8981\u624b\u6bb5\u3002\u672c\u671f\u4e50\u7ef4\u541b\u5c06\u6559\u5927\u5bb6\u5982\u4f55prometheus\u81ea\u5b9a\u4e49\u544a\u8b66\u89c4\u5219\u89e3\u6790\u548c\u914d\u7f6e\u3002<\/p>\n\n\n\n \u4ee3\u7801\u5982\u4e0b <\/p>\n\n\n\n groups: <\/p>\n\n\n\n – name: example <\/p>\n\n\n\n rules: – alert: HighErrorRate <\/p>\n\n\n\n expr: job:request_latency_seconds:mean5m{job=”myjob”} > 0.5 <\/p>\n\n\n\n for: 10m <\/p>\n\n\n\n labels: <\/p>\n\n\n\n severity: page <\/p>\n\n\n\n annotations: <\/p>\n\n\n\n summary: High request latency description: description info <\/p>\n\n\n\n \u5728\u544a\u8b66\u89c4\u5219\u6587\u4ef6\u4e2d\uff0c\u6211\u4eec\u53ef\u4ee5\u5c06\u4e00\u7ec4\u76f8\u5173\u7684\u89c4\u5219\u8bbe\u7f6e\u5b9a\u4e49\u5728\u4e00\u4e2agroup\u4e0b\u3002\u5728\u6bcf\u4e00\u4e2agroup\u4e2d\u6211\u4eec\u53ef\u4ee5\u5b9a\u4e49\u591a\u4e2a\u544a\u8b66\u89c4\u5219(rule)\u3002\u4e00\u6761\u544a\u8b66\u89c4\u5219\u4e3b\u8981\u7531\u4ee5\u4e0b\u51e0\u90e8\u5206\u7ec4\u6210\uff1a alert\uff1a\u544a\u8b66\u89c4\u5219\u7684\u540d\u79f0\u3002 <\/p>\n\n\n\n expr\uff1a\u57fa\u4e8ePromQL\u8868\u8fbe\u5f0f\u544a\u8b66\u89e6\u53d1\u6761\u4ef6\uff0c\u7528\u4e8e\u8ba1\u7b97\u662f\u5426\u6709\u65f6\u95f4\u5e8f\u5217\u6ee1\u8db3\u8be5\u6761\u4ef6\u3002<\/p>\n\n\n\n for\uff1a\u8bc4\u4f30\u7b49\u5f85\u65f6\u95f4\uff0c\u53ef\u9009\u53c2\u6570\u3002\u7528\u4e8e\u8868\u793a\u53ea\u6709\u5f53\u89e6\u53d1\u6761\u4ef6\u6301\u7eed\u4e00\u6bb5\u65f6\u95f4\u540e\u624d\u53d1\u9001\u544a\u8b66\u3002\u5728\u7b49\u5f85\u671f\u95f4\u65b0\u4ea7\u751f\u544a\u8b66\u7684\u72b6\u6001\u4e3apending\u3002 labels\uff1a\u81ea\u5b9a\u4e49\u6807\u7b7e\uff0c\u5141\u8bb8\u7528\u6237\u6307\u5b9a\u8981\u9644\u52a0\u5230\u544a\u8b66\u4e0a\u7684\u4e00\u7ec4\u9644\u52a0\u6807\u7b7e\u3002<\/p>\n\n\n\n \u4e00\u822c\u6765\u8bf4\uff0c\u5728\u544a\u8b66\u89c4\u5219\u6587\u4ef6\u7684annotations\u4e2d\u4f7f\u7528summary\u63cf\u8ff0\u544a\u8b66\u7684\u6982\u8981\u4fe1\u606f\uff0cdescription\u7528\u4e8e\u63cf\u8ff0\u544a\u8b66\u7684\u8be6\u7ec6\u4fe1\u606f\u3002\u540c\u65f6Alertmanager\u7684UI\u4e5f\u4f1a\u6839\u636e\u8fd9\u4e24\u4e2a\u6807\u7b7e\u503c\uff0c\u663e\u793a\u544a\u8b66\u4fe1\u606f\u3002\u4e3a\u4e86\u8ba9\u544a\u8b66\u4fe1\u606f\u5177\u6709\u66f4\u597d\u7684\u53ef\u8bfb\u6027\uff0cPrometheus\u652f\u6301\u6a21\u677f\u5316label\u548cannotations\u7684\u4e2d\u6807\u7b7e\u7684\u503c\u3002\u901a\u8fc7 \u53d8\u91cf\u53ef\u4ee5\u8bbf\u95ee\u5f53\u524d\u544a\u8b66\u5b9e\u4f8b\u4e2d\u6307\u5b9a\u6807\u7b7e\u7684\u503c\u3002 <\/p>\n\n\n\n $value 1 <\/p>\n\n\n\n \u5219\u53ef\u4ee5\u83b7\u53d6\u5f53\u524dPromQL\u8868\u8fbe\u5f0f\u8ba1\u7b97\u7684\u6837\u672c\u503c\u3002<\/p>\n\n\n\n \u4ee3\u7801\u5982\u4e0b<\/p>\n\n\n\n \u4f8b\u5982\uff0c\u53ef\u4ee5\u901a\u8fc7\u6a21\u677f\u5316\u4f18\u5316summary\u4ee5\u53cadescription\u7684\u5185\u5bb9\u7684\u53ef\u8bfb\u6027\uff1a<\/p>\n\n\n\n \u4ee3\u7801\u5982\u4e0b\uff1a<\/p>\n\n\n\n \u5728\u76ee\u5f55\/etc\/prometheus\/rules\/\u4e0b\u521b\u5efa\u544a\u8b66\u6587\u4ef6hoststats-alert.rules\u5185\u5bb9\u5982\u4e0b\uff1a<\/p>\n\n\n\n \u4ee3\u7801\u5982\u4e0b<\/p>\n\n\n\n \u4ee5\u4e0a\u5c31\u662fprometheus\u81ea\u5b9a\u4e49\u544a\u8b66\u89c4\u5219\u89e3\u6790\u548c\u914d\u7f6e\u7684\u5168\u90e8\u5185\u5bb9\uff0c\u5982\u679c\u5bf9\u4f60\u6709\u6240\u5e2e\u52a9\u7684\u8bdd\u8bf7\u6301\u7eed\u5173\u6ce8\u4e50\u7ef4\u5b98\u7f51<\/a>\uff0c\u4e50\u7ef4\u541b\u4f1a\u5b9a\u671f\u66f4\u65b0\u6280\u672f\u5206\u4eab\uff0c\u66f4\u591a\u5f00\u6e90\u76d1\u63a7\u6280\u672f\u4e5f\u53ef\u4ee5\u5173\u6ce8\u4e50\u7ef4\u793e\u533a<\/a>\uff08https:\/\/forum.lwops.cn\/\uff09<\/p>\n","protected":false},"excerpt":{"rendered":" \u4e0a\u4e00\u671f\u4e50\u7ef4\u541b\u8ddf\u5927\u5bb6\u5df2\u7ecf\u4ecb\u7ecd\u4e86prometheus\u7684\u5b89\u88c5\u4e0e\u914d\u7f6e\uff0c\u5bf9\u4e8e\u8fd0\u7ef4\u76d1\u63a7\u800c\u8a00\uff0c\u9664\u4e86\u76d1\u63a7\u5c55\u793a\u4ee5\u5916\uff0c\u53e6\u4e00\u4e2a\u91cd\u8981\u7684 […]<\/p>\n","protected":false},"author":1,"featured_media":11061,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"themepark_post_bcolor":"#f5f5f5","themepark_post_width":"1022px","themepark_post_img":"","themepark_post_img_po":"left","themepark_post_img_re":false,"themepark_post_img_cover":false,"themepark_post_img_fixed":false,"themepark_post_hide_title":false,"themepark_post_main_b":"","themepark_post_main_p":100,"themepark_paddingblock":false,"footnotes":""},"categories":[246],"tags":[249],"class_list":["post-11083","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-prometheus-artisan","tag-prometheus-gj"],"metadata":{"_edit_lock":["1667889239:1"],"_thumbnail_id":["11061"],"_edit_last":["1"],"themepark_seo_title":["prometheus\u81ea\u5b9a\u4e49\u544a\u8b66\u89c4\u5219\u89e3\u6790\u548c\u914d\u7f6e"],"themepark_seo_description":["\u5bf9\u4e8e\u8fd0\u7ef4\u76d1\u63a7\u800c\u8a00\uff0c\u9664\u4e86\u76d1\u63a7\u5c55\u793a\u4ee5\u5916\uff0c\u53e6\u4e00\u4e2a\u91cd\u8981\u7684\u9700\u6c42\u65e0\u7591\u5c31\u662f\u544a\u8b66\u4e86\u3002\u826f\u597d\u7684\u544a\u8b66\u53ef\u4ee5\u5e2e\u52a9\u8fd0\u7ef4\u4eba\u5458\u53ca\u65f6\u7684\u53d1\u73b0\u95ee\u9898\uff0c\u5904\u7406\u95ee\u9898\u5e76\u9632\u8303\u4e8e\u672a\u7136\uff0c\u662f\u8fd0\u7ef4\u5de5\u4f5c\u4e2d\u4e0d\u53ef\u6216\u7f3a\u7684\u91cd\u8981\u624b\u6bb5\u3002\u672c\u671f\u4e50\u7ef4\u541b\u5c06\u6559\u5927\u5bb6\u5982\u4f55prometheus\u81ea\u5b9a\u4e49\u544a\u8b66\u89c4\u5219\u89e3\u6790\u548c\u914d\u7f6e\u3002"],"themepark_seo_keyword":["prometheus,prometheus\u544a\u8b66,prometheus\u89c4\u5219"],"catce":["sidebar-widgets4"],"_yoast_wpseo_primary_category":["237"],"_yoast_wpseo_focuskw":["prometheus\u544a\u8b66\u89c4\u5219"],"_yoast_wpseo_linkdex":["41"],"_yoast_wpseo_content_score":["90"],"_yoast_wpseo_estimated-reading-time-minutes":[""],"_yoast_wpseo_wordproof_timestamp":[""],"views":["5397"],"ig_es_is_post_notified":["1"]},"yoast_head":"\n1. \u6807\u51c6\u544a\u8b66\u89c4\u5219\u6837\u4f8b\u4ee5\u53ca\u5404\u7ec4\u4ef6\u4f5c\u7528<\/h2>\n\n\n\n
2. \u6a21\u677f\u5316\u544a\u8b66\u89c4\u5219<\/h2>\n\n\n\n
$ labels.# To insert a firing element's label values:\r\n2\r\n{{ $labels.
groups:\n- name: example\nrules:\n\n# Alert for any instance that is unreachable for >5 minutes.\n- alert: InstanceDown\nexpr: up == 0\nfor: 5m\nlabels:\nseverity: page\nannotations:\nsummary: \"Instance {{ $labels.instance }} down\"\ndescription: \"{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes.\"\n\n# Alert for any instance that has a median request latency >1s.\n- alert: APIHighRequestLatency\nexpr: api_http_request_latencies_second{quantile=\"0.5\"} > 1\nfor: 10m\nannotations:\nsummary: \"High request latency on {{ $labels.instance }}\"\ndescription: \"{{ $labels.instance }} has a median request latency above 1s (current value: {{ $value }}s)\"<\/code><\/pre>\n\n\n\n
3. \u4fee\u6539Prometheus\u914d\u7f6e\u6587\u4ef6prometheus.yml<\/h2>\n\n\n\n
rule_files:\r\n- \/etc\/prometheus\/rules\/*.rules<\/code><\/pre>\n\n\n\n
groups:\n- name: hostStatsAlert\nrules:\n- alert: hostCpuUsageAlert\nexpr: sum(avg without (cpu)(irate(node_cpu{mode!='idle'}[5m]))) by (instance) > 0.85\nfor: 1m\nlabels:\nseverity: page\nannotations:\nsummary: \"Instance {{ $labels.instance }} CPU usgae high\"\ndescription: \"{{ $labels.instance }} CPU usage above 85% (current value: {{ $value }})\"\n- alert: hostMemUsageAlert\nexpr: (node_memory_MemTotal - node_memory_MemAvailable)\/node_memory_MemTotal > 0.85\nfor: 1m\nlabels:\nseverity: page\nannotations:\nsummary: \"Instance {{ $labels.instance }} MEM usgae high\"\ndescription: \"{{ $labels.instance }} MEM usage above 85% (current value: {{ $value }})\"\r<\/code><\/pre>\n\n\n\n
\u603b\u7ed3<\/h2>\n\n\n\n