JEFF BLANKENBURG

Random geekery since 2005.

Husband, father, programmer, speaker, writer, blogger, podcaster, collector, traveler, golfer.

Why Is Everyone Getting Hacked?

Written in

June 9, 2011

jeffblankenburg

In yet another example of corporate technical incompetence (or perhaps technical superiority on the part of the hackers), Citigroup announced today that “hackers accessed the data of over 200,000 bank card holders”. (full article)

I have a couple of questions about this process, because I don’t believe the PR spin.

How do they know?

Every time a large corporation gets compromised, they state, with certainty, that only certain portions of their data were leaked. “Birth dates, social security numbers, card expiration dates and card security codes (CVV) were not compromised.” How do they know that? They certainly didn’t know they had a gaping hole in their data center, how can they say with a straight face that only certain pieces of data were taken?

I suppose it’s possible that the “other data” was stored in some other database, in some other data center, but are we really supposed to believe that?

I’m by no means a security expert, but I used to run a small web server with a database. It was attacked. I can’t tell you with certainty whatsoever about what data was taken. At all. Neither could my hosting company. So how are these big companies so capable?

They say that only 1% of the 21 million North American customers were affected. How can they possibly be that certain if all of those records were sitting together in a database?

How did they fix it so quickly?

Another important part of the “we got hacked” PR is the reassurance that this won’t happen again. Like I said before, I’m not a security expert, but if I were a hacker, and I gained access to something as valuable as Citi’s database of credit card numbers, isn’t is also possible that I left a back door open so that I can come and go as I please in the future?

"We are contacting customers whose information was impacted. Citi has implemented enhanced procedures to prevent a recurrence of this type of event." If you had “enhanced procedures” immediately available to you to “prevent this type of event,” why weren’t they already implemented? How did you have exactly the solution to prevent this breach sitting around, but hadn’t done it yet?

Why did they wait so long to tell anyone?

Finally, when a breach is revealed, it’s always at least a month later. This one actually happened in early May. 30 days is an eternity in the personal data world. That’s an entire credit card billing cycle. It’s certainly long enough to borrow an identity and destroy someone’s credit rating.

My hope is that they took that time to determine the scope of the damage, but it’s far more likely that those 30 days were spent trying to figure out the best way to spin this. Much like the Russians were the scapegoats in the 80s, and the Middle East in the 90s, I find it interesting that all of the hackers seem to be living in China now. Google and Citi are pointing their fingers to the Far East. It’s far more likely that the hackers are routing their traffic through a compromised Chinese server, but I don’t want to take anything away from the 1,000,000,000+ people of China. Maybe they are just getting craftier.

Summary

Finally, I found the last paragraph of the article to be the most telling about this entire situation. Citi (and I’m sure many other organizations) seems to have thrown their hands up about security:

"Security breaches happen, they’re going to continue to happen … the mission of the banking industry is to keep the customer base safe and customers feeling secure about their financial transactions and payments."

The way I read that is this:

“We have no idea how to secure our data, but it doesn’t matter. Our responsibility is to make our customers feel like their data is safe, regardless of whether or not it really is.”

What do you think is the cause of all of these breaches at Sony, Google, and now Citi? How many hacks do we never hear about?

24 responses to “Why Is Everyone Getting Hacked?”

Damien Guard

November 29, 1999

I’m not involved in banking any more and never did any work for Citibank but I can tell you that large organisations rarely have just a few systems. They often have a network of tens to hundreds of different systems from different vendors with various pieces of glue to transfer the necessary bits of data back and forth.

If an organisation says only x number of customers were affected then those x have something in common. Maybe it was geographical or perhaps a facility that all those customers used. In this case (and I have no more information on Citibank than you) it could be something like they all redeemed or acquired reward points on one of Citibank’s cards.

These organisations will often bring in outside digital forensic specialists who analyse logs and files to figure out whether the attackers used the entry point to jumped onto another system. If you have a database of several hundred megabytes there it is often quite easy to spot it was downloaded when you see a spike in the bandwidth usage for a system that normally only deals with a few megabytes a day to and from a fixed location.

[)amien

Reply
Wayne

November 29, 1999

“Do to this, we don’t store any credit card information. If someone hacked into our database we can say no credit cards were stolen because we don’t even store it. I would imagine that they are doing something similar.”

Citibank would have a good reason to store your credit card number though. One would have to ask if they (or any merchant) are dumb enough to store a credit card number without some type of strong encryption? I know, the answer is yes, some are that dumb.

Reply
1. Bruce
  
  June 22, 2011
  
  I don’t know about how you see it, but if you have a card issued by Citi they most definitely will be storing that information since they are going to want to bill you for every transaction on that card.
  
  Reply
NNM

November 29, 1999

…And what about all the times they get hacked but NEVER noticed, and NEVER will.
Seems they only find out when some nice guy hacker decides to make the results public.
“Oh, all the emails are on pirate bay!!?”
– “Ok, call the press, tell them we got hacked!”

Reply
BlueCollarCritic

November 29, 1999

This comes down to the following simple philosophy which started by Retail to get an edge over a competitor, has grown widespread across everything like a plague and so now it has been beaten into everyone for a generation if not longer that its become second nature, something done without thought:

1) * You Deserve More for Less *
2) *You Are Not Responsible For Your Actions, it was either your parents fault or X persons fault *

So let’s see how #1 plays out in the real world.

Employee: This means they should get paid more for doing less.
Customer: This means they should get more product for less money

These 2 are in direct conflict and so both cannot be true so long as basic math (1 + 1 = 2) remains constant. If the employee gets paid more for less but the customer is also to get more while paying less then something somewhere has to get less for this to balance out let alone allow the company (that the employee works at and that the customer buys from) to make a profit which is the whole reason for taking the risk fo starting a business.

How about #2? How does this pan out in reality?

Employee: Loses job and collects unemployment (which is funded thru taxation of those who have not lost their jobs) , taking no responsibility for this job loss saying that they are ‘owed’ what they are now getting for doing nothing.
Customer: Abuses return policy and damages merchandise refusing to pay for what they (or mre likely their kids) have damaged and demanding a refund for the most outrageous of retruns.

SO after enough years of this what happens?

BUSSINESS: Offshores and finds cheaper labor elsewhere, importing its now cheaper goods back to America.
COSNUMERS: Continue lifestyle of buying stuff by opting for cheaper/lower quality products and funding purchase thru extended credit and government subsidy.

This kind of plan cannot last; a break point is coming.

How does this relate to this story? That’s easy. It’s the same problem of everyone expecting more for less. The customer paying for the software demands more for less and the software vendor accommodates this by cutting corners, sometimes to the extreme resulting in very poor quality code either thru over worked employees or from inadequately trained employees who are willing to work for less.

When you apply this “More For Less” across every industry and field is it a wonder we find ourselves living in a junk rigged system that’s just waiting for that one break point that will cause a chain reaction?

You use to be able to buy something for a reasonable amount, not cheap but not outrageously expensive, and it would last. And when it broke you would repair it (or get it repaired). Today we simply dispose of what no longer works or what we are tire of.

These 2 philosophies paired with an ever growing selfish attitude (remember when people were selfless and helped each other out?) are leading towards a collapse. Every civilization that has risen to great power has done this our current generation is very well likely the one to see it happen to ours.

Reply
Professor

November 29, 1999

Geez have a big moan.

Security is hard. Hackers will always have the upper hand because exploits take time to counter.
How do you secure your house? With a lock. Locks can be picked.
So you put in a keycode alarm. Alarms can be deactivated.
So you put bars on all the doors and windows. These can be sawn off or have locks that can be picked.
So you put in cameras… right! that’s the equivalent of ‘intrusion detection’ in network security terms. You can’t stop them but you can watch them and trace them. You can back track and trace route jumping from proxy to proxy back to where they came from, or maybe not all the way.

You can never stop someone who’s really determined. You’re not safe in your own home. But you can fingerprint the place and use other forensic clues to find the perpetrator and let justice prevail.

Reply
Celso Bressan

November 29, 1999

Although unaware of City practices (and almost all companies using computing power) and being a manager, system analyst and programmer for so many years, I can say that a good hacker could do anything he/her wants with the information available once this person accessed the system. This comes from a simple reason, once you are in, it is because you have the right credentials to be in. Therefore, these credentials grant almost if not full access to the information on the system.

I will give two reason for that:

1) Systems are designed to be integrated; if they are not, they cost a lot in terms of time and resources to keep pieces here and there;

2) A credit card user, when performing a transaction with the credit card, access pieces of software that either have full control of the data involved or needs to request other pieces of data from other places. Either way, these pieces of software MUST have access rights to all pieces, otherwise the transaction will not successfully commit.

Now, if I am a hacker and I am IN the system, this is because I have the rights AND if I am clever enough, I would fire small “SELECT *” queries and patiently collect the results over time. Usually, this does not raise suspicion in the short term, time enough to collect anything is needed to be collected.

In my humble opinion, this article hit the point correctly and there is no way to tell what went on (unless the hacker is so dumb as to leave a trail…).

The only way to avoid these situations is to prevent them. Well, then… it is all written above…

Bressan

Reply
Ian Patrick Hughes

November 29, 1999

THEY WERE HACKED BY CHANGING QUERY STRING VARIABLES?!?!?! ARE YOU EFFING KIDDING ME????? This makes me want to close my business account with them right now.

http://www.dailymail.co.uk/news/article-2003393/How-Citigroup-hackers-broke-door-using-banks-website.html

Reply
Jordan Sebastian

June 9, 2011

Yep. Sony has been hacked 10 times in the last couple months many of the breaches leaked private information. 10 times?! Really Sony? I won’t be giving them any private information for a long time.

I think the issue is that in the software development lifecycle there’s no place for security. Developers are paid and rewarded based on how quickly they close tickets and churn out code. Though there may be requirements around it being “good code” rarely does “good” include secure.

And there’s good reason for this. CEO’s don’t understand software development or IT and IT is looked at as a necessary evil to most companies. They help keep costs down but you have to invest a lot of money up front. IT departments are a lot of overhead. So pitching the idea of security reviews during software development means training developers, taking more time to get features out the door and providing the resources to do real evaluations and penetration tests. All very expensive, though in my opinion, extremely necessary.

As for the “how do they know” questions. I have some limited training in computer forensics and I know that financial institutions do an obtuse amount of logging. It wouldn’t be hard for a forensics expert to examine these logs and identify queries which didn’t (or shouldn’t have) come from the applications that interface with the databases. Odds are that no applications performs a “select * from credit_cards”.

Reply
1. Allan
  
  November 29, 1999
  
  You say that it’s the software development cycle and the developers are paid to close tickets as if that is the whole problem.
  
  First off there are these people call managers that do this thing called making decisions and who usually decide on what ‘features’ get implemented and use that to dicate that to the development team.
  
  Second off even if you manage to fully ‘secure’ the software it doesn’t mean the ‘system’ is secure. There are other avenues to hack a system and if I was a hacker and knew a website was secure I would then look at attacking it other ways (i.e. flaws in the OS, related services, employees, etc) and get access to the data via another method.
  
  Thirdly you talk about IT security being expensive but neccessary, lets see how many people are willing to pay for that extra security. These days people seem to care only about the bottom line and choosing between what seems like to similar products but one has a higher price than another, why pay the extra money.
  
  In the end it is all parties involved that need to work together to keep a system secure.
  
  1. Developers to make sure there is little in the way of security bugs.
  2. IT need to ensure the servers are secured and upto date in terms of security patches.
  3. Network needs to be secured to prevent people accessing the network and methods to alert for unusual activity.
  4. DBA’s to secure the databases and encrypt sensitive data.
  5. Employees educated enough to ensure their access can’t be used to compromise the system.
  6. The business needs proper auditing and reviews to keep a secure system up to date.
  7? Customers not to do something silly and give out sensitive information.
  
  In the end it takes a lot of energy to secure a system and one little mistake somewhere along the way can leave a big enought hole to allow a hacker to gain sensitive data.
  
  Reply
Greg Finzer

June 9, 2011

There is going to be theft in any business. Why just last month, an attendee from the Stir Trek conference stole a credit card from someone in Westerville, OH and used it on my site. Fortunately we have anti-fraud systems in place and the hacker didn’t get the software. It still cost $75 for the charge back though. BTW, hackers normally steal the CCV code on the back of the card too, so that doesn’t help. There are countries that have higher percentages of hacker attempts, China is one of them. Vietnam is high too.

Do to this, we don’t store any credit card information. If someone hacked into our database we can say no credit cards were stolen because we don’t even store it. I would imagine that they are doing something similar.

Reply
1. Robert T
  
  June 10, 2011
  
  “we can say no credit cards were stolen because we don’t even store it”
  
  I have taken this approach another big step further by eliminating the server. Granted the resulting P2P software system is complex, but it doesn’t have a target painted on its (server) back because there is none.
  
  Reply
2. Bruce
  
  June 22, 2011
  
  Citi would not be doing something similar. They are a bank; they ISSUE credit cards. They MUST maintain this information since they are the ones doing the billing of all of the charges on those cards otherwise everyone who has a card issued by them would be charging everything since it would end up being free for them. How can everyone here be missing this?
  
  Reply
Mark Freeman

June 9, 2011

My Citi credit card was declined when I tried to use it this evening. I called and was told it was on hold due to suspicious activity. They read me the charges made over the last 48 hours and half were fraudulent and in the $200 – $500 range each. They cancelled the account and are sending me a new card with a new account number.

After reading about the hacking incident, I went to their web site to change my password and it asked me a security question I know that I would never have picked. I called and they reset it for me so that I could change the password. I asked specifically if my account was affected by this hacking and was told “no, if it had there would be a big warning on my screen right now”. Must just be a weird coincidence.

Reply
Danno

June 10, 2011

I have similarly wondered about hacked companies and why they seem so certain of the damages. I concluded that people in cyber sec who know more than me would publicly shame these companies if the comments were totally ridiculous, so they must have some basis behind them. Also, being PR statements they would be intentionally vague.

Reply
SeattleC++

June 10, 2011

Programmers ignore security because they get paid the same whether they write secure code or not. Management has to make security important. Otherwise, it’s just another impediment to getting the job out on time, and managers are *always* interested in getting done on time.

Reply
Christian Sciberras

June 10, 2011

Great way starting yet another clueless article. Just because some company decided they don’t need a firewall to protect millions of users doesn’t mean everyone else does the same.

I myself am not a security researcher, however I can confidently assure you my assets have never been hacked, infected or modified in ways I didn’t want them to. This includes 3 of my personal computers, running Windows.

Finally, I’d like to point out that traffic logging is a standard practice.

Let’s put it in simple words, a robber isn’t detected by a security camera. But you know what the thief stole (or maybe who he is even) with the said camera.
Same with servers.

Reply
Dave Swersky

June 10, 2011

Credit card security is like national security. There is a legitimate concern that talking too much in the media will reveal the bank’s methods. Same goes for terrorism. Problem is, that gives the banks and government agencies free license to spin and smokescreen, which they do.

I believe the fundamental problem is that there are vanishingly few (MUCH smaller than the population of “developers”) true security experts. This leads to a cynical evaluation of the cost of security risk versus the cost of security. Just like Fight Club- they apply The Formula. Instead of hiring people to make them really secure, they invest in experts that compare the cost of screwing up to the cost of *actually securing* the data.

I worked for a company that provided web-based marketing services to Sony. Let’s just say I am not the least bit surprised by Sony’s recent breaches. Remember, every single one of these companies (banks to a lesser degree) are giving their IT contracts to the lowest bidder.

Reply
1. Katherine Jones
  
  November 29, 1999
  
  I have student loans with Citibank and had my payment taken out monthly. Since the hack that has stopped but I haven’t received any letters from them saying I failed to pay this month or last. Could these be good hackers? With a plan like that in Fight Club (to eliminate the bondage of debt). I really hope so, but in the meantime I am left wondering if I should call them and find out what’s going on, for fear one day they will demand the whole payment at once. Advice?
  
  Reply
Weekly Links– 2010_22 (50 for Web Devs & Other Geeks) :MS-Joe (Joe Stagner)

June 17, 2011

[…] Why is Everyone getting Hacked ? […]

Reply
Blankenburg On Hacks « 36 Chambers – The Legendary Journeys: Execution to the max!

June 18, 2011

[…] Jeff Blankenburg has a post wondering about the lack of security online, and how people are able to exclaim, ex post, exactly what was (and was not) done and that Everything Will Be Alright. […]

Reply
Windows 8, native apps, and HTML5: Facts and conjecture | TechRepublic

June 21, 2011

[…] and commentary Why is everyone always getting hacked? Jeff Blankenburg has a fantastic piece about the security breaches lately, and how companies are rea…. Why move to jQuery from the Ajax Control Toolkit? Joe Stagner wrote a very good analysis of why […]

Reply
Reelix

June 23, 2011

How many hacks dont you hear about?

http://zone-h.net/archive/published=0

Go there, and F5 every few seconds…

Reply
Weekly Links– 2010_22 (50 for Web Devs & Other Geeks) :MisfitGeek (Joe Stagner)

August 10, 2011

[…] Why is Everyone getting Hacked ? […]

Reply