利用 sed 和 awk 做简单日志分析

发表于 2020-06-14 更新于 2020-06-15

环境

本实验是在 macOS 下做的，Linux 下可能会稍有不同（比如 gzcat）。

日志文件

1	gzcat data.gz \| head -n 40 # data.gz 是日志文件

系统输出：

May 13 00:01:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12513]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:01:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12513]): Service exited with abnormal code: 78
May 13 00:02:12 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.pid.mdmclient.12523): Failed to bootstrap path: path = /usr/libexec/mdmclient, error = 108: Invalid path
May 13 00:04:20 BBAOMACBOOKAIR2 syslogd[113]: ASL Sender Statistics
May 13 00:05:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12535]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:05:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12535]): Service exited with abnormal code: 78
May 13 00:09:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12536]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:09:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12536]): Service exited with abnormal code: 78
May 13 00:17:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12555]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:17:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12555]): Service exited with abnormal code: 78
May 13 00:17:59 BBAOMACBOOKAIR2 syslogd[113]: ASL Sender Statistics
May 13 00:19:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12556]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:19:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12556]): Service exited with abnormal code: 78
May 13 00:21:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12560]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:21:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12560]): Service exited with abnormal code: 78
May 13 00:22:18 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.user.914945058): Service “com.apple.xpc.launchd.unmanaged.loginwindow.594” tried to register for endpoint “com.apple.tsm.uiserver” already registered by owner: com.apple.TextInputMenuAgent
May 13 00:22:49 — last message repeated 1 time —
May 13 00:23:50 BBAOMACBOOKAIR2 timed[158]: settimeofday({0x5ebacd96,0x52ddf}) == 0
May 13 00:28:05 BBAOMACBOOKAIR2 syslogd[113]: ASL Sender Statistics
May 13 00:28:07 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.ScreenSaver.Computer-Name[12564]): Service exited due to SIGKILL | sent by Computer Name[12564]
May 13 00:28:17 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:17 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:18 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:18 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:19 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:19 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:20 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:20 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:26 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.preference.displays.MirrorDisplays): Service only ran for 9 seconds. Pushing respawn out by 1 seconds.
May 13 00:28:31 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.preference.displays.MirrorDisplays): Service only ran for 4 seconds. Pushing respawn out by 6 seconds.
May 13 00:29:49 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12610]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:29:49 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12610]): Service exited with abnormal code: 78
May 13 00:30:00 BBAOMACBOOKAIR2 syslogd[113]: Configuration Notice:
ASL Module “com.apple.cdscheduler” claims selected messages.
Those messages may not appear in standard system log files or in the ASL database.
May 13 00:30:00 BBAOMACBOOKAIR2 syslogd[113]: Configuration Notice:
ASL Module “com.apple.install” claims selected messages.
Those messages may not appear in standard system log files or in the ASL database.
May 13 00:30:00 BBAOMACBOOKAIR2 syslogd[113]: Configuration Notice:
ASL Module “com.apple.callhistory.asl.conf” claims selected messages.

阅读全文 »

用 Kindle 来追网文

发表于 2020-05-01 更新于 2020-07-12

用 Kindle 来追网文

为嘛要干这个呢？这个说来话长，可能还是跟本人的实际情况有关系，别人可能还真没有这需求，毕竟，各种设备基本上都有 Kindle 和 Instapaper 的应用吧，直接看不香吗？所以，这里不解释，有需求的自然懂。

代码及主要思路来自于：这里

阅读全文 »

手撸一个 Android 的 RAT(reote administration tool)

发表于 2020-03-14 更新于 2020-03-15

整一个 Android 的 RAT

缘起

这个需求来源没法说，就是想找个 Android 手机的远程管理工具。

阅读全文 »

How to change MySQL server's data directory on Ubuntu

发表于 2020-02-04 更新于 2020-02-21

How to change MySQL server’s data directory on Ubuntu

背景

MySQL server

MySQL server 的缺省 Data 目录是 /var/lib/mysql
/var/lib/mysql 目录一般在 / 分区下
/ 区一般都不大，而且不推荐放数据
所以但凡正常 MySQL server，都存在 Data 目录放真正数据分区的需求
如果刚开始时没有设置好 Data 目录在数据分区，那么后来都有迁移 Data 目录到数据分区的需求

阅读全文 »

克里斯汀·多德瑞（Christine Dodrill）的访谈

发表于 2020-01-16 更新于 2020-01-19

克里斯汀·多德瑞（Christine Dodrill）的访谈

NOTE:

原文来自于 Christine Dodrill
我 HOST 的版本在 Christine Dodrill

您是谁，你做什么工作

我是 Christine Dodrill，西雅图地区的一位高级 SRE，目前在蒙特利尔。我曾在 Heroku，IMVU 和 Pure Storage 工作，但目前在 Lightspeed POS 工作。

阅读全文 »

How to dump all wiki pages from Phabricator

发表于 2019-12-16 更新于 2019-12-19

How to dump all wiki pages from Phabricator

参照的源代码在这里：https://gist.github.com/HackToHell/9f261728f4ceda1605c9b3b1f46addbc

install software

yum -y install \
	python-sqlalchemy \
	python-pandas \
	MySQL-python;

阅读全文 »

用 wireguard 在两个网络之间打洞

发表于 2019-11-02 更新于 2019-11-03

0 缘起

基于管理和安全的考虑，我们制定了一个大内网计划（T13），希望将公司所有的网路（办公室，各个公有云的 VPC 等）的内网打通。具体可选方案很多，我们这里采用的是 wireguard

阅读全文 »

How to auto deploy Hexo site to GitHub pages via Github Actions

发表于 2019-10-02 更新于 2019-10-03

缘起

业界良心的谷歌云（gcp）让我给玩儿坏了呀，原本这一期还没结束（应该到年底），可我的 300 刀的免费额度一个月前就已经花光了，于是乎，我的服务被停掉了。数据都没能机会拉取下来。

这样的话，给我自己折腾了进二十年的博客一个新的归宿又马上变成了火烧眉毛的事情了，早先其实一直在关注 GitHub Actions，而且也早就有申请，所以就乘着这个机会，把博客源文件、网站（静态内容）全都放在 GitHub（GitHtb Pages 服务）上吧，同时还顺带用一下 GitHub Actions 来做自动构建、部署。

阅读全文 »

服务器操作系统安装配置标准推荐

发表于 2019-09-12 更新于 2019-09-13

操作系统安装配置规范

从事运维这么长时间，每到一个新公司，都会涉及到写标准化、规范化、流程化、制度化文档的工作，这里就整理了一份，但愿对大家有帮助。

操作系统选型

以下都是推荐值，如果没有特殊的需求，请都按照以下推荐值来操作

实体机：首选 CentOS 7 系列的最新版，其次选 Ubuntu Server 的最新的 LTS 的 64 位版本，目前最新的是 18.04
阿里云：选 Aliyun Linux 2 的最新版，当下是 2.1903；选 64 位版本
AWS：选 Amazon LInux 2 的最新版；选 64 位版本

阅读全文 »

How to disable startup services from Aliyun

发表于 2019-08-13 更新于 2019-08-15

How to disable startup services from Aliyun

for i in aliyun aegis
do
    systemctl stop $i
    systemctl disable $i
done