Monday, May 05, 2008
Back-scatter spam
There was a recent slashdot post on this. We've had a fair amount of this sort of spam. And the victims are at pretty high levels of our organization, too. Last week the person who is responsible for us even having a Blackberry Enterprise Server asked us to figure out a way to prevent these emails from being forwarded to their blackberry. When a spam campaign is rolling, that person can get a bounce-message every 5-15 minutes for up to 8 hours, into the wee hours of the night. And that's just the mails that get PAST our anti-spam appliance. We set up some forwarding filters, but we haven't heard back about how effective they are.
This is a hard thing to guard against. You can't use the reputation of the sender IP address, since they're all legitimate mailers being abused by the spam campaign and are returning delivery service notices per spec. So the spam filtering has to be by content, which is a bit less effective. In one case, of the 950-odd DSN's we received for a specific person during a specific spam campaign, only 15 made it to the inbox. But that 15 was enough above what they normally saw (about 3 a day) that they complained.
Backscatter is a problem. However, our affected users have so far been sophisticated enough users of email to realize that this was more likely forgery than something wrong with their computer. So, we haven't been asked to "track down those responsible." This is a relief for us, as we've been asked that in the past when forged spams have come to the attention of higher level executives.
If it becomes a more wide-spread problem, we will be told to Do Something by the powers that be. Unfortunately, there isn't a lot that can be done. Blocking these sorts of DSNs is doable, but that's an expensive thing to manage in terms of people time. In 6-12 months we can expect the big anti-spam vendors to include options to just block DSN's uniformly, but until that time comes (and we have the budget for the added expenses) we'd have to do it through dumb keyword filters. Not a good solution. And it would also cause legitimate bounce messages to fail to arrive.
This is a hard thing to guard against. You can't use the reputation of the sender IP address, since they're all legitimate mailers being abused by the spam campaign and are returning delivery service notices per spec. So the spam filtering has to be by content, which is a bit less effective. In one case, of the 950-odd DSN's we received for a specific person during a specific spam campaign, only 15 made it to the inbox. But that 15 was enough above what they normally saw (about 3 a day) that they complained.
Backscatter is a problem. However, our affected users have so far been sophisticated enough users of email to realize that this was more likely forgery than something wrong with their computer. So, we haven't been asked to "track down those responsible." This is a relief for us, as we've been asked that in the past when forged spams have come to the attention of higher level executives.
If it becomes a more wide-spread problem, we will be told to Do Something by the powers that be. Unfortunately, there isn't a lot that can be done. Blocking these sorts of DSNs is doable, but that's an expensive thing to manage in terms of people time. In 6-12 months we can expect the big anti-spam vendors to include options to just block DSN's uniformly, but until that time comes (and we have the budget for the added expenses) we'd have to do it through dumb keyword filters. Not a good solution. And it would also cause legitimate bounce messages to fail to arrive.
Friday, April 11, 2008
On email, what comes in it
A friend recently posted the following:
Looking at statistics on the mail filter in front of Exchange, it looks like 5.9% of incoming messages for the last 7 days are clean. That is a LOT of messages getting dropped on the floor. This comes to just shy of 40,000 legitimate mail messages a day. For comparison, the number of mail messages coming in from Titian (the student email system, and unpublished backup MTA) has a 'clean' rate of 42.5%, or 2800ish legit messages a day.
People expect their email to be legitimate. Directory-harvesting attacks do constitute the majority to discrete emails; these are the messages you receive that have weird subjects, come from people you don't know, but don't have anything in the body. They're looking to see which addresses result in 'no person by that name here' messages and those that seemingly deliver. This is also why people unfortunate enough to have usernames or emails like "fred@" or "cindy@" have the worst spam problems of any organization.
As I've mentioned many times, we're actively considering migrating student email to one of the free email services offered by Google or Microsoft. This is because historically student email has had a budget of "free", and our current strategy is not working. The way it is not working is because the email filters aren't robust enough to meet expectation. Couple that with the expectation of effectively unlimited mail quota (thank you Google) and student email is no longer a "free" service. We can either spend $30,000 or more on an effective commercial anti-spam product, or we can give our email to the free services in exchange for valuable demographic data.
It's very hard to argue with economics like that.
One thing that you haven't seen yet in this article are viruses. In the last 7 days, our border email filter saw that 0.108% of incoming messages contain viruses. This is a weensy bit misleading, since the filter will drop connections with bad reputations before even accepting mail and that may very well cut down the number of reported viruses. But the fact remains that viruses in email are not the threat they once were. All the action these days are on subverted and outright evil web-sites, and social engineering (a form of virus of the mind).
This is another example of how expectation and reality differ. After years of being told, and in many cases living through the after-effects of it, people know that viruses come in email. The fact that the threat is so much more based on social engineering hasn't penetrated as far, so products aimed at the consumer call themselves anti-virus when in fact most of the engineering in them was pointed at spam filtering.
Anti-virus for email is ubiquitous enough these days that it is clear that the malware authors out there don't bother with email vectors for self-propagating software any more. That's not where the money is. The threat had moved on from cleverly disguised .exe files to cunningly wrought (in their minds) emails enticing the gullible to hit a web site that will infest them through the browser. These are the emails that border filters try to keep out, and it is a fundamentally harder problem than .exe files were.
The big commercial vendors get the success rate they do for email cleaning in part because they deploy large networks of sensors all across the internet. Each device or software-install a customer turns on can potentially be a sensor. The sensors report back to the mother database, and proprietary and patented methods are used to distill out anti-spam recipes/definitions/modules for publishing to subscribed devices and software. There is nothing saying that an open-source product can't do this, but the mother-database is a big cost that someone has to pay for and is a very key part of this spam fighting strategy. Bayesian filtering only goes so far.
And yet, people expect email to just be clean. Especially at work. That is a heavy expectation to meet.
80-90% of ALL email is directory harvesting attacks. 60-70% of the rest is spam or phishing. 1-5% of email is legit. Really makes you think about the invisible hand of email security, doesn't it?Those of us on the front lines of email security (which isn't quite me, I'm more of a field commander than a front line researcher) suspected as much. And yes, most people, nay, the vast majority, don't realize exactly what the signal-to-noise ratio is for email. Or even suspect the magnitude. I suspect that the statistic of, "80% of email is crap," is well known, but I don't think people even realize that the number is closer to, "95% of email is crap."
Looking at statistics on the mail filter in front of Exchange, it looks like 5.9% of incoming messages for the last 7 days are clean. That is a LOT of messages getting dropped on the floor. This comes to just shy of 40,000 legitimate mail messages a day. For comparison, the number of mail messages coming in from Titian (the student email system, and unpublished backup MTA) has a 'clean' rate of 42.5%, or 2800ish legit messages a day.
People expect their email to be legitimate. Directory-harvesting attacks do constitute the majority to discrete emails; these are the messages you receive that have weird subjects, come from people you don't know, but don't have anything in the body. They're looking to see which addresses result in 'no person by that name here' messages and those that seemingly deliver. This is also why people unfortunate enough to have usernames or emails like "fred@" or "cindy@" have the worst spam problems of any organization.
As I've mentioned many times, we're actively considering migrating student email to one of the free email services offered by Google or Microsoft. This is because historically student email has had a budget of "free", and our current strategy is not working. The way it is not working is because the email filters aren't robust enough to meet expectation. Couple that with the expectation of effectively unlimited mail quota (thank you Google) and student email is no longer a "free" service. We can either spend $30,000 or more on an effective commercial anti-spam product, or we can give our email to the free services in exchange for valuable demographic data.
It's very hard to argue with economics like that.
One thing that you haven't seen yet in this article are viruses. In the last 7 days, our border email filter saw that 0.108% of incoming messages contain viruses. This is a weensy bit misleading, since the filter will drop connections with bad reputations before even accepting mail and that may very well cut down the number of reported viruses. But the fact remains that viruses in email are not the threat they once were. All the action these days are on subverted and outright evil web-sites, and social engineering (a form of virus of the mind).
This is another example of how expectation and reality differ. After years of being told, and in many cases living through the after-effects of it, people know that viruses come in email. The fact that the threat is so much more based on social engineering hasn't penetrated as far, so products aimed at the consumer call themselves anti-virus when in fact most of the engineering in them was pointed at spam filtering.
Anti-virus for email is ubiquitous enough these days that it is clear that the malware authors out there don't bother with email vectors for self-propagating software any more. That's not where the money is. The threat had moved on from cleverly disguised .exe files to cunningly wrought (in their minds) emails enticing the gullible to hit a web site that will infest them through the browser. These are the emails that border filters try to keep out, and it is a fundamentally harder problem than .exe files were.
The big commercial vendors get the success rate they do for email cleaning in part because they deploy large networks of sensors all across the internet. Each device or software-install a customer turns on can potentially be a sensor. The sensors report back to the mother database, and proprietary and patented methods are used to distill out anti-spam recipes/definitions/modules for publishing to subscribed devices and software. There is nothing saying that an open-source product can't do this, but the mother-database is a big cost that someone has to pay for and is a very key part of this spam fighting strategy. Bayesian filtering only goes so far.
And yet, people expect email to just be clean. Especially at work. That is a heavy expectation to meet.
Thursday, October 25, 2007
This one leaked through
Every so often something slips by the spam filters and also catches my attention. Maybe a couple times a year, but this one needed chasing.
I got a mail on a private account with the highly suspicious subject line of "YOU HAVE WON!!!!!!!!!!!!!!"
Rightie then. Time for a text-mode reader! PINE to the rescue! I drop into header mode so it won't render anything in there. This happens fairly frequently when things leak, I like to see the header-spam to see what the spam checkers thought of it on the way through. This one was somewhat unremarkable, but one thing did stand out. It passed SPF checks.
Really? So a little wget magic and I have the file, which I crack open with strings and I get this text:
I got a mail on a private account with the highly suspicious subject line of "YOU HAVE WON!!!!!!!!!!!!!!"
Rightie then. Time for a text-mode reader! PINE to the rescue! I drop into header mode so it won't render anything in there. This happens fairly frequently when things leak, I like to see the header-spam to see what the spam checkers thought of it on the way through. This one was somewhat unremarkable, but one thing did stand out. It passed SPF checks.
X-RC-DBID: 046c9cac-dc1e-47d7-acbb-d595ac2651b6
X-RC-ID: 20071025215619610
X-RC-IP: 209.8.50.37
X-RC-FROM:
X-RC-RCPT:
DomainKey-Signature: a=rsa-sha1;
h=Received:From:To:Reply-To:Subject:MIME-Version:Content-Type:Message-Id:Dat
e;
b=e3NoRXbKhaqJoV3E9ofjd93PAw0NK64MJVN2M3AYWq2t0oDuGu9TJ/nbFp/UUyclm2BRKlf/0R
EJP05/UN9dia4UmNKmmCRlhsvg/ov0dAgbjRUktkKwWW32izAfrA3uczt6fFSjmAy3U76siqXxNH
/QlL/RWHQbX2i8KIAx0KA=; c=nofws; d=yousendit.com; q=dns; s=signed
Received: from localhost (unknown [209.8.50.53])
by wa-smtp-02.yousendit.com (Postfix) with ESMTP id 6FA7B3550334
for ; Thu, 25 Oct 2007 14:56:15 -0700 (PDT)
From: Victor Kundala via YouSendIt
To: xxxxxxxxxxxxxxx,
Reply-To: victor_kundala5@yahoo.co.uk
Subject: YOU HAVE WON!!!!!!!!!!!!!!
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="=_7f6931b42522c2a348a97f74dbe1dad0"
Message-Id: <20071025215615.6fa7b3550334@wa-smtp-02.yousendit.com>
Date: Thu, 25 Oct 2007 14:56:15 -0700 (PDT)
Hello from YouSendIt,
Hello from YouSendIt,
You have a file or files called Dear Winner.doc (1 file(s)) from
victor_kundala5@yahoo.co.uk waiting for download.
You can click on the following link to retrieve your File. The link will expire
in 14 Days .
Link: http://download.yousendit.com/05CE02D8475BB9F9
Do not reply to this automatically-generated email. If you have any questions,
please email us at paidsupport@yousendit.com.
-----
File too big for email? Try YouSendIt at @ysi.base.url@
YouSendIt
1919 S.Bascom Ave., 3rd Floor
Campbell, CA 95008Really? So a little wget magic and I have the file, which I crack open with strings and I get this text:
Dear WinnerIt's a phish! And in homage to its 409 past, it even has a Nigerian-sounding name. Awwww.
We happily announce to you today, the draw of the online UK National Lottery programme held on 20th of October 2007. Your e-mail address won you in the second category, your e-mail address attached to a ticket numbers: 4-33-34-38-39-49(bonus no.23).
You have therefore been approved to claim a total sum of
420,200 British pounds sterling. You are to contact our AFFILIATE COURIER COMPANY for delivery of your winning certificate and winning cheque.
You are to reply to this email address below: MR SOLOMON STONE INTERNATIONAL COURIER SYSTEMS EMAIL: solo_stone2004@yahoo.com Congratulations once more from all members and staffs of this programme.
Yours Truly,
Victor Kundala
Labels: spam
Thursday, March 08, 2007
Spam stats!
Yummy stats! These are from the anti-spam appliance in front of Exchange, for the last 24 hours.
And now, definitions:
Processed: The number of messages processed. This is unexploded, so that mail sent to 42 people still counts as just 1.
Spam: The number of Spam messages with a confidence of 90% or higher.
Suspected Spam: The number of Spam messages with a user defined confidence of (in this case) 70% or higher.
Attacks: An aggregate statistic, but in this case they're all Directory Harvest Attack messages. A directory-harvest-attack message is one of those messages sent to 20 people at a site with generated names, in an effort to see which addresses don't generate a bounce message.
Allowed/Blocked: We don't use this feature.
Viruses: Viruses that are not mass mailers.
Suspected Viruses: Heuristically detected viruses. Good for picking up permutations of common viri.
Worms: Viruses that are mass mailers.
Unscannable: Messages that are unscannable for whatever reason.
Like my boss, you may be looking at that 50% number and wonder what happened. It is commonly reported in the press that, "90% of all email is now spam," so where are the other 40% going? I looked into where the press were getting their numbers, and most of them get them from MessageLabs. They report their numbers on the Threat Watch. Today, the Spam rate is, "48.43%", so the 50% we're seeing is well within reason. Looking at their historical data the spam rate waxes and wanes on a day to day and week to week basis.
| Processed | Spam | Suspected Spam | Attacks | Blocked | Allowed | Viruses | Suspected Virus | Worms | Unscannable | |||||||||||||||||
| Summary |  | 168,802 | | 85,166 (50%) | | 544 (<> | | 4,837 (3%) | | 0 (0%) | | 0 (0%) | | 43 (<> | | 31 (<> | | 3,730 (2%) | | 1,772 (1%) |
And now, definitions:
Processed: The number of messages processed. This is unexploded, so that mail sent to 42 people still counts as just 1.
Spam: The number of Spam messages with a confidence of 90% or higher.
Suspected Spam: The number of Spam messages with a user defined confidence of (in this case) 70% or higher.
Attacks: An aggregate statistic, but in this case they're all Directory Harvest Attack messages. A directory-harvest-attack message is one of those messages sent to 20 people at a site with generated names, in an effort to see which addresses don't generate a bounce message.
Allowed/Blocked: We don't use this feature.
Viruses: Viruses that are not mass mailers.
Suspected Viruses: Heuristically detected viruses. Good for picking up permutations of common viri.
Worms: Viruses that are mass mailers.
Unscannable: Messages that are unscannable for whatever reason.
Like my boss, you may be looking at that 50% number and wonder what happened. It is commonly reported in the press that, "90% of all email is now spam," so where are the other 40% going? I looked into where the press were getting their numbers, and most of them get them from MessageLabs. They report their numbers on the Threat Watch. Today, the Spam rate is, "48.43%", so the 50% we're seeing is well within reason. Looking at their historical data the spam rate waxes and wanes on a day to day and week to week basis.
Saturday, March 03, 2007
Editorial: responses to the Slashdot thread
In this age, there is not much point in a school going halfway with an email system...either offer something reasonably close to the state-of-the-art or outsource it to someone who does. If you do neither, it won't get used. Even mandating the use of the school email doesn't work. You end up with professors collecting their students' gmail/hotmail/etc addresses at the beginning of the semester and having a TA type all those addresses into a mailing list.A good point. Our Fac/Staff side is done to corporate standards, and is pretty good. We use Exchange, and pay for some (rather good) anti-SPAM appliances. The quality of email provided to our FacStaff is state of the art. Student side is another matter. The prime mailer right now is handled by the venerable postfix, with antispam provided by other open-source products.
-paeanblack (191171)
In both cases, though, mail quota doesn't come even remotely close to the "gmail standard". I THINK student quota is 100MB these days, and I could be quite wrong. We have students mailing (*sigh*) 10MB power-point files around, so that can get chewed up right quick. Students get POP and IMAP support, though from what I hear the SPAM problem is the main complaint, and there is some grumbling that squirrel mail isn't the best interface to use.
You give them a campus e-mail address. It's the *official* address. Delivery to that mailbox for all official college correspondence is guaranteed. THEN, if you opt to forward it off-campus to gmail or wherever, that's your own business, and you're responsible for the failings of such at your own peril.This is what we do. The official address is the @cc.wwu.edu address. Students can then forward that mail to somewhere else if they so wish (and a lot do). We haven't accepted an off-campus 'official address' because of the inability to guarantee delivery of things like billing and assignments.
Dredd13 (14750)
I don't understand the problem with having a universal campus-hosed e-mail service. They have servers accessible to the outside world, so why not throw in an e-mail server? If you make it simple (ie: SquirrelMail seems to be a popular campus e-mail hosting app, probably cause of it's cost and simplicity), I wouldn't think size would be an issue, as long as you set the proper quotas per e-mail/user.The problem with this is funding. We use SquirrelMail. Unfortunately, the spam problem is bad enough that we need to spend money, not just admin time, to fix the problem to the end user's sastisfaction. Spending money for 18,000 accounts is not cheap by any stretch. Spending on that front is largely tied to student tech fees, which students are understandably loath to increase more than they have to. I don't know what success we've had getting fees approved for things like commercial anti-SPAM products.
-Anonymous Coward
All students will be forced onto the system by the end of the semester, but it doesn't support POP or IMAP. Because of that limitation, the only freely available mail client it supports is Windows Live Desktop, which is only available on WindowsThis is a problem that has been brought up. A sizable percentage of our student population has PowerBooks as their primary computer, and a Windows-only solution isn't workable. Our Computer Science department is, understandably, a den of anti-Microsoft sentiment (which is why the cs.wwu.edu domain receives mail independant of the central services). This is one of the reasons why we NEED something like POP or better yet IMAP support in whatever we go with. Web-only portals like gmail can work, but some students really like just dropping all their mail into a single mail client that has links to all of their email accounts.
-Topic head
I agree, switching to gmail for university email doesn't sound that bad. Especially if it would raise the storage limit from 20 MB to >2GB. I don't really care though, I almost never use my university email as I have all of my class email sent to my Yahoo/SBC account.Before the current Windows Live vs. Google debate started there were murmurings of looking at converting to a gmail setup. We got hung up on several of the points mentioned in my previous post; no SSO, no easy account create/delete, no password sync.
-assassinator42 (844848)
My University [dailynorthwestern.com] is switching to Google. One of my concerns is that I really like my desktop clients (alpine and thunderbird) and prefer IMAP. While gmail is an excellent web-client, I don't really use my gmail account that much, because it doesn't offer IMAP & POP is both "flaky" and limiting.IMAP is something of a sore point with us techs. We prefer it to POP. Neither service offers IMAP yet, which is one of the reasons we haven't lept in with glad cries.
-Anonymous Coward
You're forgetting about something, though. Microsoft give huge discounts and tons of free stuff to colleges, therefore the colleges have raging boners for Microsoft.Heh. Us more than most, since we're close enough to Redmond that a number of our alumni work for Microsoft and can donate software from the Company Store. That's how we paid for MS Office the last time around. The IRS has changed some rules to make that more expensive, but it is still a lot cheaper than regular alumni appeals. This is how we were able to afford to import all students into Active Directory.
-Anonymous Coward
However... while Microsoft is 'the cheap option' a lot of the time, recent licensing changes at Microsoft have made it much more expensive for us, and our Alumni arm-twisters. We're still wondering how we're going to pay for Exchange 2007. Vista... oof. Not going there yet. Like ALL institutions, we've factored in a certain level of money for software and Microsoft is making themselves more expensive. So, the raging boner is going flacid.
Besides, we've been a NetWare shop for a long time. Hah!
Our boss dismissed the idea of outsourcing to Google or anybody else based SOLELY upon the fact that they reserved the right to advertise in the future to our students. We don't view our students as a commodity to be sold, so that kinda killed the whole "outsource the email" idea.Yeah, that's giving us pause too. Neither outright states that they won't advertise to students. Both admit they'll be using usage data to improve their advertising targeting in general.
-Sorthum (123064)
Labels: spam
Outsourcing student e-mail
I saw on Slashdot today a piece about a University migrating their student email to Windows Live.
There have been high-level discussions about doing the same here at WWU, only we're still trying to figure out if Windows Live or a Google program makes the most sense. No decision has been made, though Windows Live would integrate much better into our environment due to the presence of student accounts in Active Directory. The Google offering has better, 'hearts and mind,' support among us techs, but the Microsoft offering would require less work from us techs to get running.
Last I heard, neither offering supported IMAP. GMail doesn't support IMAP, so I doubt any Google offer would. No idea if Windows Live (general access) even does.
There are a number of reasons why outsourcing email is attractive, and right there at the top is SPAM. We can't afford any commercial product to do student anti-spam, as they all charge per-head and even $2/head gets pretty spendy when you have to cover 18,000 student accounts. Currently, student e-mail anti-SPAM is all open-source and I still hear that the SPAM problem is pretty bad. The most senior of our unix admins spends about half his day dealing with nothing but SPAM related problems, so outsourcing would save us that expense as well.
The number two reason is price. Both the Google offering and Microsoft offering are free. Both have promised that they won't put advertising in their web portals for active students, but the usage data may be used to tailor advertising programs targeted (elsewhere) at the high-profit college-age population. Both offerings permit the student to maintain the address after graduation, though in that case they would get advertising in their web portals.
There are a number of problems that outsourcing introduces.
There have been high-level discussions about doing the same here at WWU, only we're still trying to figure out if Windows Live or a Google program makes the most sense. No decision has been made, though Windows Live would integrate much better into our environment due to the presence of student accounts in Active Directory. The Google offering has better, 'hearts and mind,' support among us techs, but the Microsoft offering would require less work from us techs to get running.
Last I heard, neither offering supported IMAP. GMail doesn't support IMAP, so I doubt any Google offer would. No idea if Windows Live (general access) even does.
There are a number of reasons why outsourcing email is attractive, and right there at the top is SPAM. We can't afford any commercial product to do student anti-spam, as they all charge per-head and even $2/head gets pretty spendy when you have to cover 18,000 student accounts. Currently, student e-mail anti-SPAM is all open-source and I still hear that the SPAM problem is pretty bad. The most senior of our unix admins spends about half his day dealing with nothing but SPAM related problems, so outsourcing would save us that expense as well.
The number two reason is price. Both the Google offering and Microsoft offering are free. Both have promised that they won't put advertising in their web portals for active students, but the usage data may be used to tailor advertising programs targeted (elsewhere) at the high-profit college-age population. Both offerings permit the student to maintain the address after graduation, though in that case they would get advertising in their web portals.
There are a number of problems that outsourcing introduces.
- Identity synchronization. MS is easiest, Google will require some custom code.
- Password synchronization. Do we even want to do it? If so, how? If not, why not?
- Account enable/disable. How do we deactivate accounts?
- Single sign-on. Is it possible to integrate whichever we use into CAS? Can we integrate it into the WWU Portal?
- Web interface skinning. Will they permit skinning with the WWU style, or will they force their own?
Labels: spam
Friday, February 02, 2007
Depressing stock spams
In the course of dealing with the new antispam appliances, I've had to look at a lot of spam. Wowzers, a lot of spam. Most of them are stock scams, which jives with the industry conventional wisdom about spam these day. On a lark, I dropped some of the symbols into CNN Money to see what suckers had fallen for the scams. Too many.
Drop 'PSUD' into your favorite stock tracker, and look at the 10 day report. I saw the mails arrive mid Monday, which is after the buy-up. Two days ago, trading volume was about 4x what it normally got. As of today, the price is still above what it was two weeks ago.
'AFML' had activity two days ago. Their chart shows a clear bump a few weeks ago where the stock was abused. The volume average is well above yesterday's volume, so this is another victim of pump-n-dump.
'QCPC' in a message from yesterday, has today's volume about 2.5x their volume average, another clear sign of pumping. They have been a victim of this scan several times over the last two months. Perhaps the scammers are trying to make back money lost?
Depressing statistics, none the less. Unlike other scams, this is something you can actually see happen from your desk. You only hear about victims of the Nigerian scams in the news, or if you're unlucky through the grapevine. Stocks are tracked WIDELY, so you can see them rise and fall.
Drop 'PSUD' into your favorite stock tracker, and look at the 10 day report. I saw the mails arrive mid Monday, which is after the buy-up. Two days ago, trading volume was about 4x what it normally got. As of today, the price is still above what it was two weeks ago.
'AFML' had activity two days ago. Their chart shows a clear bump a few weeks ago where the stock was abused. The volume average is well above yesterday's volume, so this is another victim of pump-n-dump.
'QCPC' in a message from yesterday, has today's volume about 2.5x their volume average, another clear sign of pumping. They have been a victim of this scan several times over the last two months. Perhaps the scammers are trying to make back money lost?
Depressing statistics, none the less. Unlike other scams, this is something you can actually see happen from your desk. You only hear about victims of the Nigerian scams in the news, or if you're unlucky through the grapevine. Stocks are tracked WIDELY, so you can see them rise and fall.
Labels: spam
Thursday, February 01, 2007
Community filters
The admin of the student email system related a tale the other day that I found interesting. They use dspam for their anti-spam needs, and it has a Bayesian filter. It also has some other features which have, as I said, interesting side-effects.
There is a local independent movie theater that sends out a newsletter. Some students have plonked the newsletter into Spam rather than unsubscribe.
The dspam system is configured so that if enough students mark a specific sender as spam, then that sender is blacklisted system wide.
You can see where this is going? I thought so. Enough students have reported this independent movie theater's newsletter as spam that the whole system now blocks it, and we're getting reports of 'false positive!'
There is a local independent movie theater that sends out a newsletter. Some students have plonked the newsletter into Spam rather than unsubscribe.
The dspam system is configured so that if enough students mark a specific sender as spam, then that sender is blacklisted system wide.
You can see where this is going? I thought so. Enough students have reported this independent movie theater's newsletter as spam that the whole system now blocks it, and we're getting reports of 'false positive!'
Labels: spam
Thursday, January 25, 2007
SPAM!
The decision to tell the appliances to delete Spam was made yesterday. Anything coming in flagged as Spam, not Suspect Spam, will be dropped. This is 99% of the stuff flagged as spam, as 'suspect' is a really small category. This does reduce the load on the Exchange front-end servers as they have to do much less spam checking and handle a lot fewer messages. Though, as I'll show below, only a little less data.
And now, fun stats for Yesterday!
Total messages processed: 193,242
Percentage flagged as Spam: 49%
Percentage flagged as Suspect Spam: less than 1%
Virus mails: 731 messages
Top virus: Trojan.Peacomm (45% of viruses)
Top non-WWU inbound mailer: 129.41.62.246
Top spam sender: service@watermarkcu.org, 4% of spam (go phish!)
The mail flow goes something like this:
[inbound] -> BigIP -> Appliance -> BigIP -> Exchange FrontEnd -> Exchange
The BigIP is used to load-balance between the exchange front-ends for SMTP traffic. As it flows through the BigIP, I get stats on data volume over those ports
Mail volume to Appliances: 1.7G
Mail volume to Exchange: 1.4G
So data volume isn't greatly affected by dropping 49% of incoming mail. What is affected is the number of messages being processed. The front-end servers weren't terribly loaded as it was, this just means that Outlook Web Access is more responsive than it was.
And now, fun stats for Yesterday!
Total messages processed: 193,242
Percentage flagged as Spam: 49%
Percentage flagged as Suspect Spam: less than 1%
Virus mails: 731 messages
Top virus: Trojan.Peacomm (45% of viruses)
Top non-WWU inbound mailer: 129.41.62.246
Top spam sender: service@watermarkcu.org, 4% of spam (go phish!)
The mail flow goes something like this:
[inbound] -> BigIP -> Appliance -> BigIP -> Exchange FrontEnd -> Exchange
The BigIP is used to load-balance between the exchange front-ends for SMTP traffic. As it flows through the BigIP, I get stats on data volume over those ports
Mail volume to Appliances: 1.7G
Mail volume to Exchange: 1.4G
So data volume isn't greatly affected by dropping 49% of incoming mail. What is affected is the number of messages being processed. The front-end servers weren't terribly loaded as it was, this just means that Outlook Web Access is more responsive than it was.
Labels: spam
Friday, January 12, 2007
MORE SPAM!
On days like this, I really think I should pick up this T-Shirt. I've been tempted by it for a while. Just sayin'.
That said, now that the thingy has been in place for more than 24 hours I have some interesting data to play with. Unlike previous estimates, the appliance has handled 'only' 230,000 emails in the 24 hours period defined as 9am to 9am today. This is about a fifth of previous estimates, which makes me wonder what we were counting.
What's also interesting is how few viruses have been detected. It looks like the era of the mass mailer worm is largely over. Of that 230K odd mails, only 240 viruses were found. Most of them were mass-mailers, of course, but this is not the way things were even 3 years ago.This appliance is an anti-spam appliance that also does anti-virus, not the other way around like some other appliances I can think of.
That said, now that the thingy has been in place for more than 24 hours I have some interesting data to play with. Unlike previous estimates, the appliance has handled 'only' 230,000 emails in the 24 hours period defined as 9am to 9am today. This is about a fifth of previous estimates, which makes me wonder what we were counting.
What's also interesting is how few viruses have been detected. It looks like the era of the mass mailer worm is largely over. Of that 230K odd mails, only 240 viruses were found. Most of them were mass-mailers, of course, but this is not the way things were even 3 years ago.
Thursday, January 11, 2007
New anti-spam appliance
The new anti-spam appliance finally has a license file, so I can start dorking around with it.
Happily, this appliance DOES catch picture-spam! YAY!
Unfortunately it also classifies the following as pic-spam:
I must say, it does a pretty good job. It scores on a 0-100 scale, which it sadly doesn't expose, and is hardcoded to toss anything that scores in the 90-100 range. And... it makes good decisions. You can tune the 'suspected spam' threshold lower then that, which is what I've been tweaking. Happily, it's in 'monitor and record' mode, so I can watch message flow without actually DOING anything with the messages; letting the antispam software actually on the Exchange boxes handle the load. This allows me to set the 'suspect' threshold to various spots and look to see what it tags.
Set it low enough, and I saw one message from a student to Financial Aid, asking about canceling a loan for the quarter, got picked up. Yep, raised the threshold a few ticks after that one. Apparently The Economist sends out bulletins, and that gets picked up around the 65 range. A group of students was chatting in e-mail about a class that got canceled yesterday (ice and snow), which got tagged due to the number of people on the To: line (also at about 65). One googlegroups message discussing in a scholarly way a subject that appears in spam a lot, which was tagged when the filter was set to 70.
All in all, less than 1% of the messages tagged as SPAM are tagged 'suspect'. This thing does a good job.
Happily, this appliance DOES catch picture-spam! YAY!
Unfortunately it also classifies the following as pic-spam:
To: <Everyone>Perhaps the spam/ham threshold was a bit low. Most pic-spam I know of is one line of text and an attached image. Which also makes it hard to differentiate between that stuff and stuff like this:
From: "The Bowler Family" <redacted>
Subject: In need of a serious laugh?
The Purina Diet
I was in Wal-Mart buying a large bag of Purina for my dogs and was in line to check out.
A woman behind me asked if I had a dog........ Duh!
I was feeling a bit crabby so on impulse, I told her no, I was starting The Purina Diet again, although I probably shouldn't because I'd ended up in the hospital last time, but that I'd lost 50 pounds before I awakened in an intensive care unit with tubes coming out of most of my orifices and IV's in both arms.
[...]
[attachments: "dadshirt Bkgrd.gif"]
To: YouIt's the pic-spam that is causing the powers that be to start mumbling about finding money, somewhere, anywhere, to just stop it. We've had these appliances sitting on the floor for a few months now, waiting for priorities to shift to the point where we can work with them. Now they have, and now I have.
From: Me
Subject: Too damned cute
Dickens was sleeping upside down again. This time, I got a picture.
[attachment: UpsidedownHedgehog.JPG]
I must say, it does a pretty good job. It scores on a 0-100 scale, which it sadly doesn't expose, and is hardcoded to toss anything that scores in the 90-100 range. And... it makes good decisions. You can tune the 'suspected spam' threshold lower then that, which is what I've been tweaking. Happily, it's in 'monitor and record' mode, so I can watch message flow without actually DOING anything with the messages; letting the antispam software actually on the Exchange boxes handle the load. This allows me to set the 'suspect' threshold to various spots and look to see what it tags.
Set it low enough, and I saw one message from a student to Financial Aid, asking about canceling a loan for the quarter, got picked up. Yep, raised the threshold a few ticks after that one. Apparently The Economist sends out bulletins, and that gets picked up around the 65 range. A group of students was chatting in e-mail about a class that got canceled yesterday (ice and snow), which got tagged due to the number of people on the To: line (also at about 65). One googlegroups message discussing in a scholarly way a subject that appears in spam a lot, which was tagged when the filter was set to 70.
All in all, less than 1% of the messages tagged as SPAM are tagged 'suspect'. This thing does a good job.
Labels: spam
Tuesday, October 31, 2006
Spam numbers
The following came out in the "Academic Technology News" yesterday:
Tags: spam
To put the new spam filtering in perspective, consider the following: WWU errs on the side of caution to ensure that we do not filter any legitimate email; even with this 'cautionary' configuration more than 80% of all inbound email to campus is filtered out of our email system as known spam, compared to around 65% with our previous solution. In terms of numbers, that means that the staggering number of 1.3 million spam emails are filtered from your incoming mail each day.There you have it. 1.6 million messages a day! Our Exchange system has around 6000 email accounts.
Number of emails received: 1.6 million
Number of messages filtered: 1.3 million
Number of messages delivered: 0.3 million
Tags: spam
Labels: spam
