PQA is a cool program. However, I found a few problems when parsing my log files. I actually fixed most of these, so
I think you may want to include this in your code.
Cheers,
Tadek
-> Problem 1 - Query.normalize
Normalization does not handle escaped quotes (\') properly
-> Solution 1:
Remove \' before removing text in Query.normalize - just add another gsub.
-> Problem 2 - MySQLLogLine.is_continuation
Query continuation fails to detect cases like these (actual MySQL log file)
68 Query select author_id, author_name, author_nickname, author_password, author_type,
author_email, author_url, author_can_create_blog, author_hint, author_created_by, author_can_view_log, author_public_key,
author_preferred_language, author_remote_auth_username, author_remote_auth_token
from mt_author
where author_id = '3'
041025 10:53:28 68 Query select comment_id, comment_blog_id, comment_entry_id, comment_author,
comment_commenter_id, comment_visible, comment_email,
comment_url, comment_text, comment_ip, comment_created_on, comment_created_by, comment_modified_on, comment_modified_by
from mt_comment
where (comment_entry_id = '117')
->Solution 2:
Modify the regexp to something like this
@recognized && /^(\d{1,6})|( )/.match(@text) == nil
-> Problem 3 - SyslogLogLine
Error lines from PostgreSQL are appended to queries.
-> Solution 3:
class SyslogLine:
ERROR_LINE = Regexp.new("(ERROR|FATAL|NOTICE|LOG: +recycled|LOG: +pq_recvbuf)"
def initialize(data) {
...
return if !ERROR_LINE.match(data).nil?
This works for me, but maybe you want something more general - I'm not sure what error/warning/log/etc PostgreSQL can
generate.
-> Problem 4 - SyslogAccumulator
Queries are only closed, when there's a new query with the same connection_id.
-> Solution 4
This is probably not a probem for the most frequently occurring queries, but might be a problem when you have very few
queries. Anyway, the fix would be to close the open queries when there's no more data to read: def close_out_all { #close
all open queries here } and calling it at the end of the file. I actually added it to every class so that you can call
close_out_all in GenericLogReader.
|