PgFoundry Logo
     Advanced search
Log In
New Account
  
 
Home My Page Project Tree Project Openings Practical Query Analyzer
 
 
 
 
Summary Forums Tracker News SCM Files
 
 

Bugs: Browse | Admin

[#1000138] Multiple parsing problems and fixes (both MySQL and Syslog)

Date:
2004-10-27 08:48
Priority:
3
Submitted By:
Tadeusz Pietraszek (tadekp)
Assigned To:
Tom Copeland (tom)
Category:
None
State:
Closed
Summary:
Multiple parsing problems and fixes (both MySQL and Syslog)

Detailed description
PQA is a cool program. However, I found a few problems when parsing my log files. I actually fixed most of these, so
I think you may want to include this in your code.

Cheers,
Tadek


-> Problem 1 - Query.normalize
Normalization does not handle escaped quotes (\') properly

-> Solution 1:
Remove \' before removing text in Query.normalize - just add another gsub.


-> Problem 2 - MySQLLogLine.is_continuation
Query continuation  fails to detect cases like these (actual MySQL log file)
                     68 Query       select author_id, author_name, author_nickname, author_password, author_type,
author_email, author_url, author_can_create_blog, author_hint, author_created_by, author_can_view_log, author_public_key,
author_preferred_language, author_remote_auth_username, author_remote_auth_token
from mt_author
where author_id = '3'
041025 10:53:28      68 Query       select comment_id, comment_blog_id, comment_entry_id, comment_author,
comment_commenter_id, comment_visible, comment_email,
comment_url, comment_text, comment_ip, comment_created_on, comment_created_by, comment_modified_on, comment_modified_by
from mt_comment
where (comment_entry_id = '117')


->Solution 2:
Modify the regexp to something like this
@recognized && /^(\d{1,6})|(    )/.match(@text) == nil

-> Problem 3 - SyslogLogLine
Error lines from PostgreSQL are appended to queries.

-> Solution 3:
class SyslogLine:
 ERROR_LINE = Regexp.new("(ERROR|FATAL|NOTICE|LOG: +recycled|LOG: +pq_recvbuf)"

def initialize(data) {
...
return if !ERROR_LINE.match(data).nil?

This works for me, but maybe you want something more general - I'm not sure what error/warning/log/etc PostgreSQL can
generate.


-> Problem 4 - SyslogAccumulator
Queries are only closed, when there's a new query with the same connection_id. 

-> Solution 4
This is probably not a probem for the most frequently occurring queries, but might be a problem when you have very few
queries. Anyway, the fix would be to close the open queries when there's no more data to read: def close_out_all { #close
all open queries here } and calling it at the end of the file. I actually added it to every class so that you can call
close_out_all in GenericLogReader.

Please login

Followup

No Followups Have Been Posted

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

Field Old Value Date By
close_date2005-01-15 19:242005-01-15 19:24tom
status_idOpen2005-01-15 19:24tom
assigned_tonone2004-10-30 00:10tom
summaryMultiple parsing problmes and fixes (both MySQL and Syslog)2004-10-27 08:49tadekp
summaryMultiple parsing probmes and fixes (both MySQL and Syslog)2004-10-27 08:49tadekp