Products
GG网络技术分享 2025-03-18 16:15 2
I tried to get SQL information from log files using regular expressions
The information I need is the running time of the SQL, the SQL statement, the parameters of the SQL
Here\'s my code:
package mainimport (
\"os\"
\"fmt\"
\"io/ioutil\"
\"regexp\"
)
func main(){
file,err:=os.OpenFile(\"./log.txt\",os.O_RDWR,0755)
if err !=nil{
fmt.Println(err)
}
b,err:=ioutil.ReadAll(file)
str:=string(b)
r,err:=regexp.Compile(`\\[ORM\\][\\w\\W]+?(\\d*.\\d*ms|\\d*.\\d*s)\\][ -]+\\[([\\w\\W]+?)\\]`)
s:=r.FindAllStringSubmatch(str,-1)
fmt.Println(s[0][3])
fmt.Println(s[1][3])
}
This is my log sample
[ORM]2018/08/03 10:23:50 -[Queries/read] - [ OK / db.Query / 432.4ms] - [SELECT acc.*,gp.group_name,gp.group_id,org.org_name,group_concat(r.role_name) role_name
FROM sys_account acc
LEFT JOIN sys_org org on org.org_id=acc.org_id
LEFT JOIN sys_group gp on gp.group_id=org.group_id
LEFT JOIN sys_account_role ar on ar.acct_id=acc.acct_id and ar.is_del=0
LEFT JOIN sys_role r on r.role_id=ar.role_id where 1=1 and acc.acct_type=1 group by acc.acct_id order by acc.create_time desc LIMIT 0, 15] - `1` `ASDFASDF`
nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) received CLOSE_WAIT from nsqd
nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) beginning close
nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) readLoop exiting
nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) breaking out of writeLoop
nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) writeLoop exiting
[ORM]2018/08/03 10:23:50 -[Queries/default] - [ OK / db.Query / 0.6ms] - [select * from sys_group where group_id=? ] - `111` `qwqwqw`
I hope so Print out the
`1` `ASDFASDF`
`111` `qwqwqw`
I\'m now left with the last argument that I can\'t get
These parameters may or may not be multiple、You can also have too many arguments wrapping around the file
I have tried this myself：
r,err:=regexp.Compile(`\\[ORM\\][\\w\\W]+?(\\d*.\\d*ms|\\d*.\\d*s)\\][ -]+\\[([\\w\\W]+?)\\][ -]*((\\W\\w*\\W{1,2})*)`)r,err:=regexp.Compile(`\\[ORM\\][\\w\\W]+?(\\d*.\\d*ms|\\d*.\\d*s)\\][ -]+\\[([\\w\\W]+?)\\][- ]*([^
]*)`)
You probably want something like this regexp:
var re = regexp.MustCompile(`\\[ORM\\][^-]* -\\[[^]]*\\] - \\[ .* / .* / ([^]]*)] - \\[([^]]*)\\] - (.*)`)https://play.golang.org/p/MUBzaiyyCWt
This assumes well-formed data, one entry per line. If not you probably won\'t be able to parse it with a regexp, and even this makes some assumptions about queries not including the strings [] (which you might get away with).
First I\'d double check that you absolutely cannot get this data output in the right format in the first place - if you can it\'d be far better to get the output you want directly, perhaps even have a separate log, or even better send metrics like this to a time series database for later analysis. Parsing logs for this sort of info isn\'t great, particularly if done on an ongoing basis.
If you must parse logs, some advice on testing this:
本文来源：https://github.com/ziishaned/learn-regex
作者：Zeeshan Ahmad
正则表达式是一组由字母和符号组成的特殊文本，它可以用来从文本中找出满足你想要的格式的句子。
一个正则表达式是一种从左到右匹配主体字符串的模式。 “Regular expression”这个词比较拗口，我们常使用缩写的术语“regex”或“regexp”。 正则表达式可以从一个基础字符串中根据一定的匹配模式替换文本中的字符串、验证表单、提取字符串等等。
想象你正在写一个应用，然后你想设定一个用户命名的规则，让用户名包含字符、数字、下划线和连字符，以及限制字符的个数，好让名字看起来没那么丑。 我们使用以下正则表达式来验证一个用户名：
以上的正则表达式可以接受 john_doe、jo-hn_doe、john12_as。 但不匹配Jo，因为它包含了大写的字母而且太短了。
正则表达式其实就是在执行搜索时的格式，它由一些字母和数字组合而成。 例如：一个正则表达式 the，它表示一个规则：由字母t开始，接着是h，再接着是e。
Demand feedback