Difference between revisions of "Virus:Email Attachment Blocking"

From SME Server
Jump to navigationJump to search
m (add security category)
 
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Virus & executable content blocking tutorial for qpsmtpd ==
+
{{Level|Advanced}}
 
+
==Email Attachments Virus & Executable content blocking==
This functionality has been incorporated into sme server v7.0 and more recent versions. See the Email panel in server manager.
+
The functionality to block possible executable and virus files attached to emails has been incorporated into SME Server v7.x. See the [[SME_Server:Documentation:Administration_Manual:Chapter13#E-mail_Filtering|Email]] panel in server manager.
  
 
===Problem===
 
===Problem===
 
+
Your SME Server receives a lot of email with virus infected attachments and you want to reject it before it enters your server's mail system. You want to block email with certain types of file attachments to improve security of your server or reduce bandwidth use caused by unwanted or undesired large multimedia files. Current methods typically use Anti Virus detection software, which are processor & memory intensive.
Your sme server receives a lot of email with virus infected attachments and you want to reject it before it enters your server's mail system. You want to block email with certain types of file attachments to improve security of your server or reduce bandwidth use caused by unwanted or undesired large multimedia files. Current methods typically use Anti Virus detection software, which are processor & memory intensive.
 
  
 
===Solution===
 
===Solution===
 +
This functionality allows incoming and outgoing messages to be rejected if the attached file has executable content, which matches specific file type patterns. A default pattern matching database is created with common executable file patterns, which cover the majority of currently known Windows type executable viruses. Patterns can be created for any file types to allow multimedia or other attachments to be rejected where the system management policy considers it appropriate.
  
This functionality allows incoming & outgoing messages to be rejected if the attached file has executable content, which matches specific file type patterns. A default pattern matching database is created with common executable file patterns, which cover the majority of currently known Windows type executable viruses. Patterns can be created for any file types to allow multimedia or other attachments to be rejected where the system management policy considers it appropriate.
+
Email messages are rejected if the attachment content matches an entry in the patterns database. By default this includes the majority of *.exe files, older v1.0 *.zip files and some *.gif files. This blocking applies to both incoming and outgoing smtp email messages, including the local network, in order to stop virus propagation. If these file types need to be sent using email, they should be compressed using WinZip (v2.0 format) or WinRAR or other suitable compression software, or alternatively shared on the local network use file sharing. Note that recent releases of compression software use the v2.0 zip format.
 
 
Email messages are rejected if the attachment content matches an entry in the patterns database. By default this includes the majority of *.exe files, older v1.0 *.zip files and some *.gif files. This blocking applies to both incoming and outgoing smtp email messages, including the local network, in order to stop virus propagation. If these file types need to be sent using email, they should be compressed using WinZip (v2.0 format) or WinRAR or other suitable compression software, or alternatively shared on the local network use filesharing. Note that recent releases of compression software use the v2.0 zip format.
 
  
 
Messages with attachments that match the patterns database are rejected by the mail system, and as a result there is no further processing. In practice a large number of virus infected messages will be rejected, perhaps 95 % or more, depending on the type of virus infections you receive and your system exposure (email addresses).
 
Messages with attachments that match the patterns database are rejected by the mail system, and as a result there is no further processing. In practice a large number of virus infected messages will be rejected, perhaps 95 % or more, depending on the type of virus infections you receive and your system exposure (email addresses).
  
In conjunction with RBL list blocking of spam messages you can expect a reduction in virus detections by Clamavis from typically hundreds per month to one message per month. The use of RBL list spam blocking also helps reduce virus infected email messages entering the server, probably due to the fact that some virus infected messages come from similar sources as spam messages.
+
In conjunction with Real-time Blackhole List (RBL) blocking of spam messages you can expect the reduction in virus detections by ClamAV is from typically hundreds per month to one message per month. The use of RBL list spam blocking also helps reduce virus infected email messages entering the server, probably due to the fact that some virus infected messages come from similar sources as spam messages.
  
This method works for servers configured as either Server & Gateway or Server Only as long as the mail server components are enabled (qpsmtp & qmail) and the server has access to the Internet via another sme server or firewall.
+
This method works for servers configured as either Server & Gateway or Server Only as long as the mail server components are enabled (qpsmtp & qmail) and the server has access to the Internet via another SME Server or firewall.
  
 
===Additional Information===
 
===Additional Information===
 
 
Pattern matching acts as a "gross filter" to reject many known virus types, but a regularly updated virus scanner is still required to catch new viruses. Once these new executable content viruses have been analysed, additional patterns can be created and added to the patterns database as required. It is envisaged that new patterns would be added infrequently.
 
Pattern matching acts as a "gross filter" to reject many known virus types, but a regularly updated virus scanner is still required to catch new viruses. Once these new executable content viruses have been analysed, additional patterns can be created and added to the patterns database as required. It is envisaged that new patterns would be added infrequently.
  
 
This pattern matching feature should be used in conjunction with virus scanning software and spam filtering software, although these programs will have a lot less work to do. Pattern blocking should be compatible with other brands of virus & spam software based programs. They generally scan or filter the message after it has been accepted by the servers mail system. Pattern blocking occurs before the message is accepted, and if a matching occurs the message is rejected so it would never be scanned by secondary software based systems. Incompatibilities are therefore unlikely.
 
This pattern matching feature should be used in conjunction with virus scanning software and spam filtering software, although these programs will have a lot less work to do. Pattern blocking should be compatible with other brands of virus & spam software based programs. They generally scan or filter the message after it has been accepted by the servers mail system. Pattern blocking occurs before the message is accepted, and if a matching occurs the message is rejected so it would never be scanned by secondary software based systems. Incompatibilities are therefore unlikely.
  
An additional feature I recommend to implement is "RBL List blocking" using qpsmtp, to reject spam messages from senders that are included on RBL lists. This technique will dramatically reduce the amount of spam entering the server. Additional patterns can be added to the database after install is completed. Also see separate section below for information on analysing, creating & adding patterns.
+
Additional patterns can be added to the database after install is completed. Also see separate section below for information on analysing, creating & adding patterns.
  
WARNING: Enable additional patterns with care. Verify that the patterns do not block attachment types that you wish to receive.
+
An additional feature recommended to implement is [[:SME_Server:Documentation:FAQ#Real-time_Blackhole_List_.28RBL.29|RBL List blocking]] using qpsmtp, to reject spam messages from senders that are included on RBL lists. This technique will dramatically reduce the amount of spam entering the server. You should also consider the additional [[:SME_Server:Documentation:FAQ#Spam|Spam Blocking]] measures generally
  
+
{{Warning box|Enable additional patterns with care. Verify that the patterns do not block attachment types that you wish to receive.}}
  
 
===Enabling Pattern Matching===
 
===Enabling Pattern Matching===
 
+
There is a menu box in the server manager [[SME_Server:Documentation:Administration_Manual:Chapter13#E-mail_Filtering|Email panel]] which allows executable content blocking to be enabled or disabled. It is disabled by default. Use "Ctrl click" to highlight or un-highlight the various groups of file types, and then click the Save button to enable/disable pattern matching.
A menu box is in the server manager Email panel, which allows executable content blocking to be enabled or disabled. It is disabled by default. Use "Ctrl click" to highlight or unhighlight the various groups of file types, and then click the Save button to enable/disable pattern matching.
 
  
 
===Analyzing and creating patterns===
 
===Analyzing and creating patterns===
  
 
====Common file patterns (or signatures or magic)====
 
====Common file patterns (or signatures or magic)====
 
 
The standard patterns enabled by default are:
 
The standard patterns enabled by default are:
  
 
Windows executables seen in active viruses
 
Windows executables seen in active viruses
TVqQAAMAA
+
*TVqQAAMAA
TVpQAAIAA
+
*TVpQAAIAA
  
 
Additional Windows executable signatures not yet seen in viruses
 
Additional Windows executable signatures not yet seen in viruses
TVpAALQAc
+
*TVpAALQAc
TVpyAXkAX
+
*TVpyAXkAX
TVrmAU4AA
+
*TVrmAU4AA
TVrhARwAk
+
*TVrhARwAk
TVoFAQUAA
+
*TVoFAQUAA
TVoAAAQAA
+
*TVoAAAQAA
TVoIARMAA
+
*TVoIARMAA
TVouARsAA
+
*TVouARsAA
TVrQAT8AA
+
*TVrQAT8AA
TVoAAAEAAA
+
*TVoAAAEAAA
  
 
ZIP file signature seen in SoBig.E and mydoom
 
ZIP file signature seen in SoBig.E and mydoom
UEsDBAoAA (this pattern is blocked - zip v1.0 format)
+
*UEsDBAoAA (this pattern is blocked - zip v1.0 format)
UEsDBBQAA (this pattern is NOT blocked by default - zip v2.0 format
+
*UEsDBBQAA (this pattern is NOT blocked by default - zip v2.0 format
 
+
{{Note box|msg=If you block the zip v2.0 attachments, e-mail with OpenOffice documents (.odt) attached might be blocked as well, see also [[bugzilla:6045]].}}
 
GIF file found in a previous Microsoft virus
 
GIF file found in a previous Microsoft virus
R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy
+
*R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy
 
 
Extra patterns not included in default database that may be enabled if required for blocking of new viruses
 
 
 
A recent pattern identified for the virus
 
 
 
Worm.SomeFool.P
 
 
 
is
 
  
TVoAAD8AA
+
A recent pattern identified for the Worm.SomeFool.P virus
 +
*TVoAAD8AA
  
Identified as MS-DOS executable
+
(Identified as MS-DOS executable)
  
 
 
====Extra patterns====
 
====Extra patterns====
Extra patterns not included in default database that may be enabled if required for blocking of multimedia files etc (long & short versions listed)
+
Extra patterns not included in the default database that may be enabled if required for blocking of multimedia files etc (long & short versions listed)
 
+
{{Note box|These have not been thoroughly tested and may need further refinement to ensure they accurately represent the signature pattern for all occurrences of the particular file type}}
Note that these have not been thoroughly tested and may need further refinement to ensure they accurately represent the signature pattern for all occurrences of the particular file type
 
 
 
  
 
SCR screen saver files - MS-DOS executable (EXE)
 
SCR screen saver files - MS-DOS executable (EXE)
 
+
*Example: TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Example: TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAA
+
*Pattern: TVqQAAMAAA
 
 
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
 
Pattern: TVqQAAMAAA
 
 
 
 
  
 
PIF1 - data
 
PIF1 - data
 
+
*Example: AHhUYXggMTk5OCAgICAgICAgICAgICAgICAgICAgICCAAgAAWTpcSFNPRlRcSFQ5OFxIVDk4LkVY
Example: AHhUYXggMTk5OCAgICAgICAgICAgICAgICAgICAgICCAAgAAWTpcSFN
+
*Pattern: AHhUYXgg
 
 
PRlRcSFQ5OFxIVDk4LkVY
 
 
 
Pattern: AHhUYXgg
 
 
 
 
  
 
PIF2 - data
 
PIF2 - data
Example: AMlIbDk5LmV4ZSAgICAgICAgICAgICAgICAgICAgICCAAIAAVDpccH
+
*Example: AMlIbDk5LmV4ZSAgICAgICAgICAgICAgICAgICAgICCAAIAAVDpccHJpdmF0ZVxIc29mdFxITFxI
 
+
*Pattern: AMlIbDk5Lm
JpdmF0ZVxIc29mdFxITFxI
 
Pattern: AMlIbDk5Lm
 
 
 
 
  
 
PIF3 - data
 
PIF3 - data
 
+
*Example: AHhIYW5kaVJlZ2lzdGVyIDIwMDAgICAgICAgICAgICCAAgAAWTpcSHNvZnRcSFJcSFIwMC5FWEUA
Example: AHhIYW5kaVJlZ2lzdGVyIDIwMDAgICAgICAgICAgICCAAgAAWTpcSHNvZn
+
*Pattern: AHhIYW5k
 
 
RcSFJcSFIwMC5FWEUA
 
 
 
Pattern: AHhIYW5k
 
 
 
 
  
 
WAV sound file - data
 
WAV sound file - data
Example: UklGRiRwLgBXQVZFZm10IBAAAAABAAIAgLsAAADuAgAEABAAZGF0YQB
+
*Example: UklGRiRwLgBXQVZFZm10IBAAAAABAAIAgLsAAADuAgAEABAAZGF0YQBwLgAAAAAAAAAAAAAAAAAA
 
+
*Pattern: Uk1GRiRwL
wLgAAAAAAAAAAAAAAAAAA
 
 
 
Pattern: Uk1GRiRwL
 
 
 
 
  
 
JPEG image data, JFIF standard 0.00, aspect ratio, 0 x 0
 
JPEG image data, JFIF standard 0.00, aspect ratio, 0 x 0
Example: /9j/4AAQSkZJRgABAgEBLAEsAAD/7RLSUGhvdG9zaG9wIDMuMAA4QklNA+0
+
*Example: /9j/4AAQSkZJRgABAgEBLAEsAAD/7RLSUGhvdG9zaG9wIDMuMAA4QklNA+0AAAAAABABLAAAAAEA
 
+
*Pattern: /9j/4AAQSkZJRg
AAAAAABABLAAAAAEA
 
Pattern: /9j/4AAQSkZJRg
 
 
 
  
 
TIF - TIFF image data, little-endian
 
TIF - TIFF image data, little-endian
Example: SUkqAAgAAAAQAP4ABAABAAAAAAAAAAABAwABAAAAJgMAAAEBAwABAAAA
+
*Example: SUkqAAgAAAAQAP4ABAABAAAAAAAAAAABAwABAAAAJgMAAAEBAwABAAAAQAUAAAIBAwADAAAAzgAA
 
+
*Pattern: SUkqAAgAAAA
QAUAAAIBAwADAAAAzgAA
 
 
 
Pattern: SUkqAAgAAAA
 
 
 
 
  
 
PPT powerpoint presentation -Microsoft Office Document
 
PPT powerpoint presentation -Microsoft Office Document
Example: 0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAA
+
*Example: 0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAADEAwAAIRgBAAAAAAAA
 
+
*Pattern: 0M8R4KGxGuEA
DEAwAAIRgBAAAAAAAA
 
 
 
Pattern: 0M8R4KGxGuEA
 
 
 
  
 
WMV Windows Media Player video file - Microsoft ASF
 
WMV Windows Media Player video file - Microsoft ASF
Example: MCaydY5mzxGm2QCqAGLObH8PAAAAAAAACwAAAAECodyrjEepzxGO5
+
*Example: MCaydY5mzxGm2QCqAGLObH8PAAAAAAAACwAAAAECodyrjEepzxGO5ADADCBTZWgAAAAAAAAAeeIB
 
+
*Pattern: MCaydY5mzxGm
ADADCBTZWgAAAAAAAAAeeIB
 
 
 
Pattern: MCaydY5mzxGm
 
 
 
  
 
MPG mpeg1 video file - MPEG system stream data
 
MPG mpeg1 video file - MPEG system stream data
 
+
*Example: AAABuiEAAQAHgCgdAAABuwAMgCgdBeH/4OAuwMAgAAAB4AfcYC4xAAGMUREAAXAxAAABsxYBIIME
Example: AAABuiEAAQAHgCgdAAABuwAMgCgdBeH/4OAuwMAgAAAB4AfcYC4xAA
+
*Pattern: AAABuiEAAQAHg
 
 
GMUREAAXAxAAABsxYBIIME
 
Pattern: AAABuiEAAQAHg
 
 
 
  
 
M2P mpeg2 video file - MPEG system stream data
 
M2P mpeg2 video file - MPEG system stream data
 
+
*Example: AAABukQABAAGBQFG//gAAAG7AAyAo38F4X/g4OfAwCAAAAHgB9qAwQ0xAAG2QxEAAZojHmDnAAAB
Example: AAABukQABAAGBQFG//gAAAG7AAyAo38F4X/g4OfAwCAAAAHgB9qAw
+
*Pattern: AAABukQABAAGB
 
 
Q0xAAG2QxEAAZojHmDnAAAB
 
Pattern: AAABukQABAAGB
 
 
 
  
 
AVI video file - RIFF (little-endian) data
 
AVI video file - RIFF (little-endian) data
 +
*Example: UklGRpC0qQBBVkkgTElTVDYBAABoZHJsYXZpaDgAAABAnAAA5MJnAAAAAAAQAAEAWggAAAAAAAAC
 +
*Pattern: UklGRpC0qQBB
  
Example: UklGRpC0qQBBVkkgTElTVDYBAABoZHJsYXZpaDgAAABAnAAA5MJn
+
===Determining file pattern, signature or magic===
 
+
To find out what the pattern or signature or magic for a file is, it needs to be run through a base64 encoding routine and the appropriate strings determined from the first line of the output. That is, for "sane" files which have "magic" numbers at the start. The file can also be decoded to find out what type of it is. Published file specifications (where available) could also be referred to.
AAAAAAAQAAEAWggAAAAAAAAC
 
Pattern: UklGRpC0qQBB
 
 
 
===Determining file pattern signature or magic===
 
 
 
To find out what the pattern or signature or magic for a file is, we need to run it through a base64 encoding routine and look for the appropriate strings in the first line of the output. That is, for "sane" files which have "magic" numbers at the start. We also can decode the file to find out what type of file it is. Published file specifications (where available) could also be referred to.
 
  
Copy a file to a folder on your sme server, say filename.zip
+
Copy a file to a folder on SME Server, say ''filename.zip''
  
 
At the command prompt do
 
At the command prompt do
 +
perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' <filename.zip | head -1
  
perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' <filename.zip | head -1
+
This gives an output of
 
+
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV
we get an output of
 
 
 
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV
 
 
 
We need to pick a suitable substring to use as the pattern for this file type, for example:
 
 
 
UEsDBAoAA
 
 
 
We want the pattern string to be long enough to avoid "false positives" and short enough to catch all of that file type. Running the above command across a few files of a particular type will usually clearly show the appropriate substring.
 
 
 
 
  
To find out the file type details do
+
A suitable substring needs to be picked to use as the pattern for this file type, for example:
 +
UEsDBAoAA
  
 +
The pattern string needs to be long enough to avoid "false positives" and short enough to catch all of that file type. Running the above command across a few files of a particular type will usually clearly show the appropriate substring.
  
echo 'UEsDBAoAA' | perl -MMIME::Base64 -0777 -ne 'print decode_base64($_)' >/tmp/17.exe
+
To find out the file type details
 +
echo 'UEsDBAoAA' | perl -MMIME::Base64 -0777 -ne 'print decode_base64($_)' >/tmp/17.exe
  
 
then run "file" on the result
 
then run "file" on the result
 
+
file /tmp/17.exe
file /tmp/17.exe
 
  
 
the output is
 
the output is
 
+
/tmp/17.exe: Zip archive data, at least v1.0 to extract
/tmp/17.exe: Zip archive data, at least v1.0 to extract
 
  
 
which identifies the type of file
 
which identifies the type of file
 
 
 
 
  
 
An alternative way of identifying the file pattern or signature for users of Clamavis-ng is to view the quarantined messages in /var/spool/amavis-ng/quarantine
 
An alternative way of identifying the file pattern or signature for users of Clamavis-ng is to view the quarantined messages in /var/spool/amavis-ng/quarantine
 
  
 
Here is an extract from a quarantined infected message that mimics a zip file
 
Here is an extract from a quarantined infected message that mimics a zip file
 +
File: 406a8bee~aad.msg Col 0 30787 bytes
 +
----------mtohkeqkmfnipbfntepj
 +
Content-Type: application/octet-stream; name="AttachedFile.zip"
 +
Content-Transfer-Encoding: base64
 +
Content-Disposition: attachment; filename="AttachedFile.zip"
  
File: 406a8bee~aad.msg Col 0 30787 bytes
+
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV1OwspplLsSWrbYvwOvHVHYOYDOiVliyLlDWU2LYVELdEiwxkwOPVsk3+m/Ddl9U56v6+tbrdXPEBTv+yEH56h/R+Bbk54hUOLieVPW61QOD7YVXZilxgCAZ+SppPxWuKv2iCBuw5qQ5N/r7CISrWWEPaAzGYwUmuERoNMEo4TFm6yV2BqBhv+Y1e/SLz30EV6anGmvwvKiWaLfcjo8sfF3UDQ203TAV33kypvZDqAsF/g3O1rvbEf+K/pZpWjOy1A5S3OWF7IKsbNxQdwqWPvuO6XS6QHwLQAF+6q4LKdUFM89j+lnKR3bXaGU3v18YN862XIeJtEqW3Ulbj8MA33IBDoTQzpYQwGQm+?????????..
----------mtohkeqkmfnipbfntepj
 
Content-Type: application/octet-stream; name="AttachedFile.zip"
 
Content-Transfer-Encoding: base64
 
Content-Disposition: attachment; filename="AttachedFile.zip"
 
  
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV1OwspplLsSWrbYvwOvHVHYOYDOiVliyLlDWU2LYVELdEiwxkwOPVsk3+m/Ddl9U56v6+tbrdXPEBTv+yEH56h/R+Bbk54hUOLieVPW61QOD7YVXZilxgCAZ+SppPxWuKv2iCBuw5qQ5N/r7CISrWWEPaAzGYwUmuERoNMEo4TFm6yV2BqBhv+Y1e/SLz30EV6anGmvwvKiWaLfcjo8sfF3UDQ203TAV33kypvZDqAsF/g3O1rvbEf+K/pZpWjOy1A5S3OWF7IKsbNxQdwqWPvuO6XS6QHwLQAF+6q4LKdUFM89j+lnKR3bXaGU3v18YN862XIeJtEqW3Ulbj8MA33IBDoTQzpYQwGQm+?????????..
+
So to create a new pattern for this message use
 +
UEsDBAoAA
  
 
So to create a new pattern for this message we would use
 
UEsDBAoAA
 
 
which is the pattern corresponding to ZIPV1 file type
 
which is the pattern corresponding to ZIPV1 file type
 +
UEsDBAoAA: Zip archive data, at least v1.0 to extract
  
UEsDBAoAA: Zip archive data, at least v1.0 to extract
+
===Enabling or disabling patterns===
 +
====Mailpatterns DB File====
 +
The definitions and patterns etc for the various file types are stored in the SME Server configuration database file ''mailpatterns''. The property fields in the database for each defined file type are:
 +
*pattern key - the type of the entry in the database (currently only the "pattern" type is used)
 +
*Body - the substring to match
 +
*Description - free format text to describe this pattern. This text will be used to display a menu of patterns to enable/disable in a later version
 +
*Glob - whether to apply a wildcard match after the pattern
 +
*LineStart - whether to only match this pattern at the start of the line
 +
*Status - whether this pattern is currently enabled (i.e. blocked)
  
 +
In general, to add a pattern to the database for a file with pattern <Signature> and file type <XYZ>, and enable it
 +
/sbin/e-smith/db mailpatterns set <Signature> pattern Body <Signature> Description "<XYZ> file <Signature>)" Glob yes LineStart yes Status enabled
 +
signal-event email-update
  
====Enabling or disabling patterns====
+
To disable the pattern do:
 
+
/sbin/e-smith/db mailpatterns setprop <FILETYPE> Status disabled
Let's say we want to add a pattern to the existing EXEFILES type (which you should do if you discover new patterns for common new MSDOS Executable type viruses)
+
signal-event email-update
 
 
A pattern analysed from email messages received is
 
 
 
TVoAAD8AA
 
 
 
As we wish to add this to the existing db entry we would do:
 
 
 
/sbin/e-smith/db mailpatterns set EXEFILES pattern Body TVqQAAMAA,TVpQAAIAA,
 
 
 
TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,
 
 
 
TVouARsAA,TVrQAT8AA,TVoAAAEAAA,TVoAAD8AA Description "MS-DOS executables"
 
 
 
Glob yes LineStart yes Status enabled
 
  
(the above is all on one line, watch the spaces are correct)
+
Some specific examples follow.
  
 +
====Executable Type Files====
 +
To add a pattern to the existing Executable type files (which should be done if new patterns are discovered for common new MSDOS Executable type viruses)
  
signal-event email-update
+
A pattern analysed from an email message received is
 +
TVoAAD8AA
  
   
+
To add this to the db entries and enable it do:
 +
  /sbin/e-smith/db mailpatterns set TVoAAD8AA pattern Body TVoAAD8AA Description "PC executables (TVoAAD8AA)" Glob yes LineStart yes Status enabled
 +
  signal-event email-update
  
 
To check the entry is correct do:
 
To check the entry is correct do:
 
+
/sbin/e-smith/db mailpatterns show TVoAAD8AA
db mailpatterns show EXEFILES
 
  
 
which gives an output of
 
which gives an output of
 +
TVoAAD8AA=pattern
 +
    Body=TVoAAD8AA
 +
    Description=PC executables (TVoAAD8AA)
 +
    Glob=yes
 +
    LineStart=yes
 +
    Status=enabled
  
EXEFILES=pattern
+
====PIF2 Type Files====
 
+
To enable the pattern for PIF2 type files (which should be done to block some PIF attachments)
Body=TVqQAAMAA,TVpQAAIAA,TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,TVouARsAA,TVrQAT8AA,TVoAAAEAAA,TVoAAD8AA
 
 
 
Description=MS-DOS executables
 
 
 
Glob=yes
 
 
 
LineStart=yes
 
 
 
Status=enabled
 
 
 
 
 
 
Let's say we want to enable the pattern for PIF2 type files (which you should do if you wish to block some PIF attachments)
 
  
 
A pattern being tested for this file type is
 
A pattern being tested for this file type is
 +
AMlIbDk5Lm
  
AMlIbDk5Lm
+
To add this to the db entries and enable it do:
 
+
/sbin/e-smith/db mailpatterns set AMlIbDk5Lm pattern Body AMlIbDk5Lm Description "PIF2 file (AMlIbDk5Lm)" Glob yes LineStart yes Status enabled
So we do:
+
  signal-event email-update
 
 
db mailpatterns set PIF2 pattern Body AMlIbDk5Lm Description "PIF2 data" Glob yes LineStart yes Status enabled
 
 
 
(the above is all on one line)
 
 
 
 
 
signal-event email-update
 
  
 
To check the entry is correct do:
 
To check the entry is correct do:
 
+
/sbin/e-smith/db mailpatterns show PIF2
db mailpatterns show PIF2
 
  
 
which gives an output of
 
which gives an output of
 +
AMlIbDk5Lm=pattern
 +
    Body=AMlIbDk5Lm
 +
    Description=PIF2 file (AMlIbDk5Lm)
 +
    Glob=yes
 +
    LineStart=yes
 +
    Status=enabled
  
PIF2=pattern
+
====Modifying the default database====
 
+
An alternative approach is to modify the default patterns loaded in the configuration database.
Body=AMlIbDk5Lm
 
 
 
Description=PIF2 data
 
 
 
Glob=yes
 
 
 
LineStart=yes
 
 
 
Status=enabled
 
 
 
 
 
 
The fields are as follows:
 
 
 
pattern - the type of the entry in the database (currently only the "pattern" type is used)
 
 
 
Body - the substring to match
 
 
 
Description - free format text to describe this pattern. This text will be used to display a menu
 
 
 
of patterns to enable/disable in a later version
 
 
 
Glob - whether to apply a wildcard match after the pattern
 
 
 
LineStart - whether to only match this pattern at the start of the line
 
 
 
Status - whether this pattern is currently enabled (i.e. blocked)
 
 
 
 
 
 
To disable the pattern do:
 
 
 
db mailpatterns setprop PIF2 Status disabled
 
  
signal-event email-update
+
''initialize-default-databases'' loads the db with fragments from ''/etc/e-smith/db''. When new patterns are added to the master rpm, new fragments are also added.
  
   
+
Taking the PIF2 example above, to add a pattern to the default set do:
 +
  mkdir -p /etc/e-smith/db/mailpatterns/defaults/AMlIbDk5Lm/
  
+
and in that directory, create the following files with content shown:
 
+
type               pattern
The alternative but more correct approach is as follows:
+
Body               AMlIbDk5Lm
 
+
Description       PIF2 file (AMlIbDk5Lm)
The initialize-default-databases loads the db with fragments from /etc/e-smith/db. When new patterns are added to the master rpm, new fragments are also added.
+
Glob               yes
 
+
Status             enabled
To add a pattern to the default set in the rpm, we do:
 
 
 
mkdir -p /etc/e-smith/db/mailpatterns/defaults/PIF2/
 
 
 
and in that directory, create the following files/contents:
 
 
 
type/pattern
 
Body/AMlIbDk5Lm
 
Description/PIF2 data
 
Glob/yes
 
Status/enabled
 
  
 
then do
 
then do
 
+
/etc/e-smith/events/actions/initialize-default-databases
/etc/e-smith/events/actions/initialize-default-databases
 
 
 
  
 
which will load the default settings
 
which will load the default settings
 
 
  
 
To show all the patterns in the mailpatterns database & their status (enabled or disabled) do
 
To show all the patterns in the mailpatterns database & their status (enabled or disabled) do
 +
/sbin/e-smith/db mailpatterns show
  
db mailpatterns show
+
===Checking logs===
 
+
Check logs for the effectiveness of blocking messages with executable content in the attachments
which will give an output similar to
 
 
 
(Note the last entry for ZIPV2 is disabled)
 
 
 
db mailpatterns show
 
 
 
EXEFILES=pattern
 
 
 
Body=TVqQAAMAA,TVpQAAIAA,TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,TVouARsAA,TVrQAT8AA,TVoAAAEAAA
 
 
 
Description=MS-DOS executables
 
 
 
Glob=yes
 
 
 
LineStart=yes
 
 
 
Status=enabled
 
 
 
GIF01=pattern
 
 
 
Body=R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy
 
 
 
Description=GIF file from old virus
 
 
 
Glob=yes
 
 
 
LineStart=yes
 
 
 
Status=enabled
 
 
 
ZIPV1=pattern
 
 
 
Body=UEsDBAoAA
 
 
 
Description=Zip archive data, at least v1.0 to extract
 
 
 
Glob=yes
 
 
 
LineStart=yes
 
 
 
Status=enabled
 
 
 
ZIPV2=pattern
 
 
 
Body=UEsDBBQAA
 
 
 
Description=Zip archive data, at least v2.0 to extract
 
 
 
Glob=yes
 
 
 
LineStart=yes
 
 
 
Status=disabled
 
 
 
 
 
 
 
 
 
 
 
 
===Checking logs for effectiveness of blocking messages with executable content in the attachments===
 
 
 
By reviewing /var/log/qpsmtpd/current and var/log/qpsmtpd/* you can see the entries for rejected messages and generally enough information as to why the rejection occurred, and therefore see the effectiveness of Pattern Matching blocking.
 
  
Note that you will only see these types of entries after blocking has been enabled and messages have been rejected.
+
By reviewing ''/var/log/qpsmtpd/current'' and ''/var/log/qpsmtpd/*'' entries for rejected messages can be seen with generally enough information as to why the rejection occurred, and therefore the effectiveness of Pattern Matching blocking.
 +
{{Note box|Entries will only be seen after blocking has been enabled and messages have been rejected.}}
  
If you do not see all of the types of entries shown below, it would either be due to not having the particular Pattern enabled or not receiving attachments with that type of content.
+
If all of the types of entries shown below are not seen, it will either be due to not having the particular Pattern enabled or not receiving attachments with that type of content.
 
 
You can view date formatted logs using the Server Manager View log files panel
 
  
 +
Date formatted logs can be viewed using the Server Manager ''View log files'' panel
 
   
 
   
 
 
To see ALL the log entries do
 
To see ALL the log entries do
 
+
grep "" /var/log/qpsmtpd/current | tai64nlocal
grep "" /var/log/qpsmtpd/current | tai64nlocal
 
 
 
 
  
 
To see only the rejected message entries and the reason for rejection do
 
To see only the rejected message entries and the reason for rejection do
 
+
grep "We don't accept email with executable content" /var/log/qpsmtpd/current | tai64nlocal
grep "We don't accept email with executable content" /var/log/qpsmtpd/current | tai64nlocal
 
 
 
(the above is all on one line)
 
  
 
Here is an example of some typical entries
 
Here is an example of some typical entries
 +
2004-04-15 12:32:11.892522500 qpsmtp[23392]: 554 We don't accept email with executable content ZIPV1 (#5.3.4)
 +
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 +
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE12 (#5.3.4)
 +
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content PIF (#5.3.4)
  
Note: you will only see these entries after some messages have been rejected
+
Alternatively filter on the pattern type code to see how many messages with a particular type of executable content are being rejected
 
+
grep EXE01 /var/log/qpsmtpd/current | tai64local
2004-04-15 12:32:11.892522500 qpsmtp[23392]: 554 We don't accept email with executable content ZIPV1 (#5.3.4)
 
 
 
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
 
 
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE12 (#5.3.4)
 
 
 
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content PIF (#5.3.4)
 
 
 
 
  
Alternatively you could filter on the pattern type code to see how many messages with a particular type of executable content are being rejected eg
+
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
+
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE01 (#5.3.4)
grep EXE01 /var/log/qpsmtpd/current | tai64local
+
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
+
2004-04-15 15:33:24.986426500 qpsmtp[29274]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
 
 
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
 
 
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
 
 
2004-04-15 15:33:24.986426500 qpsmtp[29274]: 554 We don't accept email with executable content EXE01 (#5.3.4)
 
 
 
 
  
 
===Web sites for background information===
 
===Web sites for background information===
 +
These links may be of interest. Note that they do not specifically apply to SME Server, so DO NOT implement them. They are listed for background information only.
  
These links may be of interest. Note that they do not specifically apply to sme server, so DO NOT implement them. They are listed for background information only.
+
*http://qmail.planetmirror.com/top.html
 +
*http://qmail.planetmirror.com/top.html#microsoft
 +
*http://qmail.planetmirror.com/qmail-smtpd-viruscan-1.3.patch
  
http://qmail.planetmirror.com/top.html
+
===Prior version of this Howto for sme6.x===
 +
Here is a link to an earlier [http://mirror.contribs.org/smeserver/contribs/rmitchell/smeserver/howto/Virus%20and%20file%20blocking%20HOWTO%20using%20smtpfront-qmail%20for%20sme%20server.htm old HowTo] written for sme6.x.
  
http://qmail.planetmirror.com/top.html#microsoft
+
{{Note box| Disregard all references to installing rpms as this does not apply to SME 7.x.
 
 
http://qmail.planetmirror.com/qmail-smtpd-viruscan-1.3.patch
 
 
 
 
 
===Prior version of this Howto for sme6.x===
 
  
Here is a link to an earlier Howto written for sme6.x.
+
There are some small changes in database arrangement between the older SME 6.x db and the SME 7.x db.
Please disregard all references to installing rpms as this does not apply to sme 7.x.
 
There are some small changes in database arrangement between the older sme6.x db and the sme7.x db. Note also that there are many additional patterns in sme7.x. Note also that sme7.x uses qpsmtpd instead of smtpfront-qmail.
 
  
You will find it here [http://mirror.contribs.org/smeserver/contribs/rmitchell/smeserver/howto/Virus%20and%20file%20blocking%20HOWTO%20using%20smtpfront-qmail%20for%20sme%20server.htm old HowTo].
+
There are many additional patterns in SME 7.x. Note also that SME 7.x uses qpsmtpd instead of smtpfront-qmail.}}
  
 
----
 
----
 
[[Category:Howto]]
 
[[Category:Howto]]
 
[[Category:Mail]]
 
[[Category:Mail]]
 +
[[Category:Administration:Content Spam Virus Blocking]]
 +
[[Category:Security]]

Latest revision as of 22:11, 8 October 2014

PythonIcon.png Skill level: Advanced
The instructions on this page may require deviations from standard procedures. A good understanding of linux and Koozali SME Server is recommended.


Email Attachments Virus & Executable content blocking

The functionality to block possible executable and virus files attached to emails has been incorporated into SME Server v7.x. See the Email panel in server manager.

Problem

Your SME Server receives a lot of email with virus infected attachments and you want to reject it before it enters your server's mail system. You want to block email with certain types of file attachments to improve security of your server or reduce bandwidth use caused by unwanted or undesired large multimedia files. Current methods typically use Anti Virus detection software, which are processor & memory intensive.

Solution

This functionality allows incoming and outgoing messages to be rejected if the attached file has executable content, which matches specific file type patterns. A default pattern matching database is created with common executable file patterns, which cover the majority of currently known Windows type executable viruses. Patterns can be created for any file types to allow multimedia or other attachments to be rejected where the system management policy considers it appropriate.

Email messages are rejected if the attachment content matches an entry in the patterns database. By default this includes the majority of *.exe files, older v1.0 *.zip files and some *.gif files. This blocking applies to both incoming and outgoing smtp email messages, including the local network, in order to stop virus propagation. If these file types need to be sent using email, they should be compressed using WinZip (v2.0 format) or WinRAR or other suitable compression software, or alternatively shared on the local network use file sharing. Note that recent releases of compression software use the v2.0 zip format.

Messages with attachments that match the patterns database are rejected by the mail system, and as a result there is no further processing. In practice a large number of virus infected messages will be rejected, perhaps 95 % or more, depending on the type of virus infections you receive and your system exposure (email addresses).

In conjunction with Real-time Blackhole List (RBL) blocking of spam messages you can expect the reduction in virus detections by ClamAV is from typically hundreds per month to one message per month. The use of RBL list spam blocking also helps reduce virus infected email messages entering the server, probably due to the fact that some virus infected messages come from similar sources as spam messages.

This method works for servers configured as either Server & Gateway or Server Only as long as the mail server components are enabled (qpsmtp & qmail) and the server has access to the Internet via another SME Server or firewall.

Additional Information

Pattern matching acts as a "gross filter" to reject many known virus types, but a regularly updated virus scanner is still required to catch new viruses. Once these new executable content viruses have been analysed, additional patterns can be created and added to the patterns database as required. It is envisaged that new patterns would be added infrequently.

This pattern matching feature should be used in conjunction with virus scanning software and spam filtering software, although these programs will have a lot less work to do. Pattern blocking should be compatible with other brands of virus & spam software based programs. They generally scan or filter the message after it has been accepted by the servers mail system. Pattern blocking occurs before the message is accepted, and if a matching occurs the message is rejected so it would never be scanned by secondary software based systems. Incompatibilities are therefore unlikely.

Additional patterns can be added to the database after install is completed. Also see separate section below for information on analysing, creating & adding patterns.

An additional feature recommended to implement is RBL List blocking using qpsmtp, to reject spam messages from senders that are included on RBL lists. This technique will dramatically reduce the amount of spam entering the server. You should also consider the additional Spam Blocking measures generally


Warning.png Warning:
Enable additional patterns with care. Verify that the patterns do not block attachment types that you wish to receive.


Enabling Pattern Matching

There is a menu box in the server manager Email panel which allows executable content blocking to be enabled or disabled. It is disabled by default. Use "Ctrl click" to highlight or un-highlight the various groups of file types, and then click the Save button to enable/disable pattern matching.

Analyzing and creating patterns

Common file patterns (or signatures or magic)

The standard patterns enabled by default are:

Windows executables seen in active viruses

  • TVqQAAMAA
  • TVpQAAIAA

Additional Windows executable signatures not yet seen in viruses

  • TVpAALQAc
  • TVpyAXkAX
  • TVrmAU4AA
  • TVrhARwAk
  • TVoFAQUAA
  • TVoAAAQAA
  • TVoIARMAA
  • TVouARsAA
  • TVrQAT8AA
  • TVoAAAEAAA

ZIP file signature seen in SoBig.E and mydoom

  • UEsDBAoAA (this pattern is blocked - zip v1.0 format)
  • UEsDBBQAA (this pattern is NOT blocked by default - zip v2.0 format
Important.png Note:
If you block the zip v2.0 attachments, e-mail with OpenOffice documents (.odt) attached might be blocked as well, see also bugzilla:6045.


GIF file found in a previous Microsoft virus

  • R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy

A recent pattern identified for the Worm.SomeFool.P virus

  • TVoAAD8AA

(Identified as MS-DOS executable)

Extra patterns

Extra patterns not included in the default database that may be enabled if required for blocking of multimedia files etc (long & short versions listed)

Important.png Note:
These have not been thoroughly tested and may need further refinement to ensure they accurately represent the signature pattern for all occurrences of the particular file type


SCR screen saver files - MS-DOS executable (EXE)

  • Example: TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  • Pattern: TVqQAAMAAA

PIF1 - data

  • Example: AHhUYXggMTk5OCAgICAgICAgICAgICAgICAgICAgICCAAgAAWTpcSFNPRlRcSFQ5OFxIVDk4LkVY
  • Pattern: AHhUYXgg

PIF2 - data

  • Example: AMlIbDk5LmV4ZSAgICAgICAgICAgICAgICAgICAgICCAAIAAVDpccHJpdmF0ZVxIc29mdFxITFxI
  • Pattern: AMlIbDk5Lm

PIF3 - data

  • Example: AHhIYW5kaVJlZ2lzdGVyIDIwMDAgICAgICAgICAgICCAAgAAWTpcSHNvZnRcSFJcSFIwMC5FWEUA
  • Pattern: AHhIYW5k

WAV sound file - data

  • Example: UklGRiRwLgBXQVZFZm10IBAAAAABAAIAgLsAAADuAgAEABAAZGF0YQBwLgAAAAAAAAAAAAAAAAAA
  • Pattern: Uk1GRiRwL

JPEG image data, JFIF standard 0.00, aspect ratio, 0 x 0

  • Example: /9j/4AAQSkZJRgABAgEBLAEsAAD/7RLSUGhvdG9zaG9wIDMuMAA4QklNA+0AAAAAABABLAAAAAEA
  • Pattern: /9j/4AAQSkZJRg

TIF - TIFF image data, little-endian

  • Example: SUkqAAgAAAAQAP4ABAABAAAAAAAAAAABAwABAAAAJgMAAAEBAwABAAAAQAUAAAIBAwADAAAAzgAA
  • Pattern: SUkqAAgAAAA

PPT powerpoint presentation -Microsoft Office Document

  • Example: 0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAADEAwAAIRgBAAAAAAAA
  • Pattern: 0M8R4KGxGuEA

WMV Windows Media Player video file - Microsoft ASF

  • Example: MCaydY5mzxGm2QCqAGLObH8PAAAAAAAACwAAAAECodyrjEepzxGO5ADADCBTZWgAAAAAAAAAeeIB
  • Pattern: MCaydY5mzxGm

MPG mpeg1 video file - MPEG system stream data

  • Example: AAABuiEAAQAHgCgdAAABuwAMgCgdBeH/4OAuwMAgAAAB4AfcYC4xAAGMUREAAXAxAAABsxYBIIME
  • Pattern: AAABuiEAAQAHg

M2P mpeg2 video file - MPEG system stream data

  • Example: AAABukQABAAGBQFG//gAAAG7AAyAo38F4X/g4OfAwCAAAAHgB9qAwQ0xAAG2QxEAAZojHmDnAAAB
  • Pattern: AAABukQABAAGB

AVI video file - RIFF (little-endian) data

  • Example: UklGRpC0qQBBVkkgTElTVDYBAABoZHJsYXZpaDgAAABAnAAA5MJnAAAAAAAQAAEAWggAAAAAAAAC
  • Pattern: UklGRpC0qQBB

Determining file pattern, signature or magic

To find out what the pattern or signature or magic for a file is, it needs to be run through a base64 encoding routine and the appropriate strings determined from the first line of the output. That is, for "sane" files which have "magic" numbers at the start. The file can also be decoded to find out what type of it is. Published file specifications (where available) could also be referred to.

Copy a file to a folder on SME Server, say filename.zip

At the command prompt do

perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' <filename.zip | head -1

This gives an output of

UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV

A suitable substring needs to be picked to use as the pattern for this file type, for example:

UEsDBAoAA

The pattern string needs to be long enough to avoid "false positives" and short enough to catch all of that file type. Running the above command across a few files of a particular type will usually clearly show the appropriate substring.

To find out the file type details

echo 'UEsDBAoAA' | perl -MMIME::Base64 -0777 -ne 'print decode_base64($_)' >/tmp/17.exe

then run "file" on the result

file /tmp/17.exe

the output is

/tmp/17.exe: Zip archive data, at least v1.0 to extract

which identifies the type of file

An alternative way of identifying the file pattern or signature for users of Clamavis-ng is to view the quarantined messages in /var/spool/amavis-ng/quarantine

Here is an extract from a quarantined infected message that mimics a zip file

File: 406a8bee~aad.msg Col 0 30787 bytes
----------mtohkeqkmfnipbfntepj
Content-Type: application/octet-stream; name="AttachedFile.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="AttachedFile.zip"
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV1OwspplLsSWrbYvwOvHVHYOYDOiVliyLlDWU2LYVELdEiwxkwOPVsk3+m/Ddl9U56v6+tbrdXPEBTv+yEH56h/R+Bbk54hUOLieVPW61QOD7YVXZilxgCAZ+SppPxWuKv2iCBuw5qQ5N/r7CISrWWEPaAzGYwUmuERoNMEo4TFm6yV2BqBhv+Y1e/SLz30EV6anGmvwvKiWaLfcjo8sfF3UDQ203TAV33kypvZDqAsF/g3O1rvbEf+K/pZpWjOy1A5S3OWF7IKsbNxQdwqWPvuO6XS6QHwLQAF+6q4LKdUFM89j+lnKR3bXaGU3v18YN862XIeJtEqW3Ulbj8MA33IBDoTQzpYQwGQm+?????????..

So to create a new pattern for this message use

UEsDBAoAA

which is the pattern corresponding to ZIPV1 file type

UEsDBAoAA: Zip archive data, at least v1.0 to extract

Enabling or disabling patterns

Mailpatterns DB File

The definitions and patterns etc for the various file types are stored in the SME Server configuration database file mailpatterns. The property fields in the database for each defined file type are:

  • pattern key - the type of the entry in the database (currently only the "pattern" type is used)
  • Body - the substring to match
  • Description - free format text to describe this pattern. This text will be used to display a menu of patterns to enable/disable in a later version
  • Glob - whether to apply a wildcard match after the pattern
  • LineStart - whether to only match this pattern at the start of the line
  • Status - whether this pattern is currently enabled (i.e. blocked)

In general, to add a pattern to the database for a file with pattern <Signature> and file type <XYZ>, and enable it

/sbin/e-smith/db mailpatterns set <Signature> pattern Body <Signature> Description "<XYZ> file <Signature>)" Glob yes LineStart yes Status enabled
signal-event email-update

To disable the pattern do:

/sbin/e-smith/db mailpatterns setprop <FILETYPE> Status disabled
signal-event email-update

Some specific examples follow.

Executable Type Files

To add a pattern to the existing Executable type files (which should be done if new patterns are discovered for common new MSDOS Executable type viruses)

A pattern analysed from an email message received is

TVoAAD8AA

To add this to the db entries and enable it do:

/sbin/e-smith/db mailpatterns set TVoAAD8AA pattern Body TVoAAD8AA Description "PC executables (TVoAAD8AA)" Glob yes LineStart yes Status enabled
 signal-event email-update

To check the entry is correct do:

/sbin/e-smith/db mailpatterns show TVoAAD8AA

which gives an output of

TVoAAD8AA=pattern
    Body=TVoAAD8AA
    Description=PC executables (TVoAAD8AA)
    Glob=yes
    LineStart=yes
    Status=enabled

PIF2 Type Files

To enable the pattern for PIF2 type files (which should be done to block some PIF attachments)

A pattern being tested for this file type is

AMlIbDk5Lm

To add this to the db entries and enable it do:

/sbin/e-smith/db mailpatterns set AMlIbDk5Lm pattern Body AMlIbDk5Lm Description "PIF2 file (AMlIbDk5Lm)" Glob yes LineStart yes Status enabled
 signal-event email-update

To check the entry is correct do:

/sbin/e-smith/db mailpatterns show PIF2

which gives an output of

AMlIbDk5Lm=pattern
    Body=AMlIbDk5Lm
    Description=PIF2 file (AMlIbDk5Lm)
    Glob=yes
    LineStart=yes
    Status=enabled

Modifying the default database

An alternative approach is to modify the default patterns loaded in the configuration database.

initialize-default-databases loads the db with fragments from /etc/e-smith/db. When new patterns are added to the master rpm, new fragments are also added.

Taking the PIF2 example above, to add a pattern to the default set do:

mkdir -p /etc/e-smith/db/mailpatterns/defaults/AMlIbDk5Lm/

and in that directory, create the following files with content shown:

type               pattern
Body               AMlIbDk5Lm
Description        PIF2 file (AMlIbDk5Lm)
Glob               yes
Status             enabled

then do

/etc/e-smith/events/actions/initialize-default-databases

which will load the default settings

To show all the patterns in the mailpatterns database & their status (enabled or disabled) do

/sbin/e-smith/db mailpatterns show

Checking logs

Check logs for the effectiveness of blocking messages with executable content in the attachments

By reviewing /var/log/qpsmtpd/current and /var/log/qpsmtpd/* entries for rejected messages can be seen with generally enough information as to why the rejection occurred, and therefore the effectiveness of Pattern Matching blocking.

Important.png Note:
Entries will only be seen after blocking has been enabled and messages have been rejected.


If all of the types of entries shown below are not seen, it will either be due to not having the particular Pattern enabled or not receiving attachments with that type of content.

Date formatted logs can be viewed using the Server Manager View log files panel

To see ALL the log entries do

grep "" /var/log/qpsmtpd/current | tai64nlocal

To see only the rejected message entries and the reason for rejection do

grep "We don't accept email with executable content" /var/log/qpsmtpd/current | tai64nlocal

Here is an example of some typical entries

2004-04-15 12:32:11.892522500 qpsmtp[23392]: 554 We don't accept email with executable content ZIPV1 (#5.3.4)
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE12 (#5.3.4)
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content PIF (#5.3.4)

Alternatively filter on the pattern type code to see how many messages with a particular type of executable content are being rejected

grep EXE01 /var/log/qpsmtpd/current | tai64local
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:24.986426500 qpsmtp[29274]: 554 We don't accept email with executable content EXE01 (#5.3.4)

Web sites for background information

These links may be of interest. Note that they do not specifically apply to SME Server, so DO NOT implement them. They are listed for background information only.

Prior version of this Howto for sme6.x

Here is a link to an earlier old HowTo written for sme6.x.


Important.png Note:
Disregard all references to installing rpms as this does not apply to SME 7.x.

There are some small changes in database arrangement between the older SME 6.x db and the SME 7.x db.

There are many additional patterns in SME 7.x. Note also that SME 7.x uses qpsmtpd instead of smtpfront-qmail.