Dominic White - Masters

Thesis Patching

nospam@example.com (Dominic White) — Mon, 05 Feb 2007 23:43:59 +0200

With the help of Jason, my thesis has undergone some major grammar surgery. The new version is available at the usual place. There have been over 32K worth of changes since I received the commentary back from the examiners. If you're interested in patch management, take a read.

Monthly Patch Release Schedules: Do the Risks outweigh the Benefits?

nospam@example.com (Dominic White) — Wed, 19 Jul 2006 15:23:33 +0200

At the Information Security South Africa conference 2006 I published a paper arguing that our current understanding of the risks associated with monthly patch release cycles is pretty poor. This discussion is pretty important given that entities such as Gartner recon monthly release will be the new industry standard.

I basically argue that in the case of delayed (responsible) disclosure patch schedules work well, but in the case of instantaneous (0day) disclosure none of the purported benefits, namely better quality patches and better deployment scheduling are accrued. I then move onto some solutions.

I think this is a really important paper and a really important discussion. Of course, I am the author so I would think that. The paper is available at:

http://singe.za.net/masters/files/issa2006/issa-2006-patch_schedule.pdf

Limiting Vulnerability Exposure through effective Patch Management - a thesis

nospam@example.com (Dominic White) — Sun, 02 Jul 2006 13:55:52 +0200

My Masters thesis on Patch Management entitled "Limiting Vulnerability Exposure through effective Patch Management: threat mitigation through vulnerability remediation" is now available. Given some bandwidth problems please rather download a compressed version (zip, gzip, bzip2).

Here's the abstract:

This document aims to provide a complete discussion on vulnerability and patch management. The first chapters look at the trends relating to vulnerabilities, exploits, attacks and patches. These trends describe the drivers of patch and vulnerability management and situate the discussion in the current security climate. The following chapters then aim to present both policy and technical solutions to the problem. The policies described lay out a comprehensive set of steps that can be followed by any organisation to implement their own patch management policy, including practical advice on integration with other policies, managing risk, identifying vulnerability, strategies for reducing downtime and generating metrics to measure progress. Having covered the steps that can be taken by users, a strategy describing how best a vendor should implement a related patch release policy is provided. An argument is made that current monthly patch release schedules are inadequate to allow users to most effectively and timeously mitigate vulnerabilities. The final chapters discuss the technical aspect of automating parts of the policies described. In particular the concept of 'defense in depth' is used to discuss additional strategies for 'buying time' during the patch process. The document then goes on to conclude that in the face of increasing malicious activity and more complex patching, solid frameworks such as those provided in this document are required to ensure an organisation can fully manage the patching process. However, more research is required to fully understand vulnerabilities and exploits. In particular more attention must be paid to threats, as little work as been done to fully understand threat-agent capabilities and activities from a day to day basis.

Here is a brief chapter breakdown:

Introduction
Vulnerability and Patch Management - an analysis of vulnerability, malware and threat trends followed up by an analysis of problems with patches.
Policy Solutions - an in-depth patch management framework for creating an organisational patch management policy.
Vendor Release Patch Policy - an analysis of how vendors can best manage the risks associated with releasing patches.
Practical Solutions - an analysis of where technology is needed in patch management and what is currently available.
Conclusion

~~The thesis is still being examined after which I will submit the final version with corrections. What this means is that if you have any corrections, please send them to me.~~
The thesis was examined and passed. Irritatingly, one examiner strongly recommended a distinction while the other strongly argued against one (on the basis that the thesis was not scientific enough, but highly practical). At least there was no ambivalence.

UPDATE: added compressed versions.

UPDATE II: added new version with over 32K worth of corrections.

UPDATE III: Updated examination status to passed.

My Sword of Damocles

nospam@example.com (Dominic White) — Wed, 28 Jun 2006 20:00:49 +0200

I just finished my thesis! Woo hoo! At first it felt like giving birth, but now it feels like I just excised a cancer. What a slog. Will post links to it when I have them in a 'proper' place.

It is 208 pages, approx. 54 000 words with 251 references. This is me excited.

Security Researcher Fluffing

nospam@example.com (Dominic White) — Sun, 11 Jun 2006 21:11:14 +0200

In my soon-to-be-published paper, I make a point that it is a good idea for vendors to make friends with security researchers in an effort to encourage delayed disclosure (some people call it 'responsible' disclosure).

It is interesting then to see that Microsoft will be throwing a party for security researchers at BlackHat. This, along with their BlueHat efforts is a very good idea. I look forward to seeing if it pays off given the past (and somewhat current) negative opinion of some security practitioners towards Microsoft. Or, more simply, will it have a material effect on the number of Microsoft 0days?

ISSA 2006 Paper Accepted

nospam@example.com (Dominic White) — Wed, 31 May 2006 07:27:00 +0200

My ISSA paper was just accepted as a full research paper. The comments were pretty good too, of course I am only quoting the good bits, but:

Reviewer One:

Excellent insight shown, well researched, very relevant topic.

Reviewer Two:

The paper presents an interesting and well-written discussion, which is extensively supported by references to existing literature.

Reviewer one had mostly grammatical corrections, but reviewer two built some positive arguments against some of my points, which is always a good sign of a thoughtful reviewer and meaty arguments. I think I can rebut them pretty easily and will add them to the paper.

Rhodes is sending down a well sized phalanx of presenters, and I will be proudly representing my company. I can't wait. I just hope those lazy Sensepost bums contribute something this year, instead of recruiting ;)

Less Patch Scope == Faster Response

nospam@example.com (Dominic White) — Tue, 18 Apr 2006 07:57:36 +0200

Wow, it seems Microsoft managed to get their MS06-015 cumulative IE patch rolled out with only a few compatibility problems with older HP, NVIDIA, Siebel and Kerio Firewall products. Pretty good given the non-security ActiveX change they bundled in there.

Oh, they also fixed that security vulnerability that was activley exploited in the wild since March 23rd. Now given the lag time in patch deployment (current research suggests 19 days for internal machines), it should just be just over a month that attackers have been able to wade through the average windows box.

Can someone tell me why Microsoft decided that the best way to get a patch out as quickly as possible was to bundle a huge, non-security modificcation into it?

Information Security South Africa 2006 Abstract

nospam@example.com (Dominic White) — Fri, 10 Mar 2006 15:15:15 +0200

Here is the abstract of the paper I submitted to ISSA 2006 today. It's mostly cut & paste from the introduction to one of my thesis chapters. I really should hand that in sometime. After Ben Nagy pointed out the awful flaws in my last attempt, I came up with some better arguments, but you only get to see those at or after the conference. I think they're pretty good.

Monthly Patch Release Schedules: Do the Benefits Outweigh the Risks?

Dominic White, Deloitte

Barry Irwin, Rhodes University

Effective policies are not only the responsibility of the users of software (end-users), software vendors must have a clear understanding of how they manage the patches they release and the best way to release them. Historically vulnerability disclosure and responding to vulnerabilities has proved difficult to standardise, with a high level of confusion and antagonism between security researchers and vendors. To combat this and ensure meaningful and useful interaction between researchers and vendors several disclosure policies have been suggested; a resource dedicated to collecting publications related to disclosure lists a total of twenty two different disclosure policies published between 1999 and 2004 [1] by vendors, security researchers and third parties. This confusion makes it difficult for vendors to standardise on a release policy, and instead the responsibility for formulating an effective patch management policy is passed onto the end-user. As will be demonstrated in this paper, this is because the type of disclosure has an impact on the effectiveness of a patch release policy.

In an effort to ease the administrative burden of patching on end-users some vendors have decided to move to a predictable patch release schedule. The first vendor to announce such a move was Microsoft. Soon afterwards Oracle and Adobe announced they would also move to a predictable cycle. John Pescatore of Gartner believes predictable patch release schedules are on their way to becoming and industry standard [2]. However, simplifying a patch release cycle ignores the complexities that the full disclosure debate has introduced. In both Microsoft and Oracle's case, the reactions to the announcements were varied. Some security experts were for the move, others against and the majority were silent, the lack of consensus indicated a shortage of research and understanding as to the possible effects. Since then both Microsoft and Oracle have both come under heavy criticism, and received praise for their patch schedule implementations by security professionals commenting on the same events. Propagating this policy to other vendors without a thorough analysis and with little understanding of the effects would not be desirable.

Surface observations of the implemented schedule have revealed both successes and failures. This paper provides a detailed argumentative analysis of patch release schedules, and their effectiveness. By examining examples of how various types of disclosure affects the risks faced by end-users, recommendations on how patch schedules should be implemented and when they are effective, or not, are formulated. In addition, lessons learned from recent public security incidents are used to suggest additional improvements to the process. The resulting observations are used to describe a method for other vendors to implement such a cycle that will both minimise risk and help ease the burden of patching on administrators.

REFERENCES

[1] Vulnerability disclosure publications and discussion tracking. University of Oulu, Electrical and Information Engineering Department (May 10, 2005).

Available at: http://www.ee.oulu.fi/research/ouspg/sage/disclosure-tracking/

[2] McMillan, Robert. Adobe Adopts Monthly Patch Cycle. IDG News Service (December 15, 2005).

Available at: http://www.pcworld.com/resource/article/0,aid,123935,pg,

Shavlik and the mound of FUD

nospam@example.com (Dominic White) — Sun, 22 Jan 2006 22:09:30 +0200

I would like to nominate Mary Landesman and Shavlik for the "Quality FUD" award, for their paper entitled Security Patch Management: Breaking New Ground. The subtitle is "A discussion of agent and agentless technology", although how this relates to the title is beyond me.

In this eight page paper with precisely five references (of which one, while using the word 'agent' is clearly referring to autonomous semantic-web based agents and not client/server architecture agents) she manages to make a plethora of unsubstantiated claims plainly meant to sell Shavlik's agentless technology. Her main ploy is to make it sound like installing agents to every machine is hard work, and so having to do that every time there is a patch emergency would be bad. Of course you only need to install them once, but lets not confuse 'facts' with the truth. She doesn't seem to realise that if her Shavlik software can deploy executable patch content it could probably deploy agents too, *sssh* don't tell.

This isn't a bash at Shavlik software or an endorsement of agent based solutions. She even quotes her CEO as saying he thinks the whole debate is a "red herring", I prefer the term "a load of crap". If you are logging into a box remotely with administrator rights, then you aren't doing much different from an agent, the code just happens to be transmitted every time instead of stored locally.

I wonder how many morons they fool with this faux-academic ninja-marketing technique?

Non-M$ Patch Management

nospam@example.com (Dominic White) — Fri, 20 Jan 2006 23:03:01 +0200

Someone on the patchmanagement mailing list just asked how you do patch management on FreeBSD and OpenBSD! This has been a bugbear of mine for a while. Microsoft isn't good at patching, they are only just catching up. Personally I feel Debian and FreeBSD are the current industry leaders in the patch management field. Let me tell you why.

First off, very few people in the unix world call it patch management. It is usually called package management. There is a difference between the terms patch and package management. Package management is a subset of patch management that specifically focuses on deployment, patch notification, handling dependencies and the like. Patch management usually includes broader activities like network inventorys and vulnerability scanning as well as package management. Thus in the context of this discussion we are actually talking about package management tools.

The open-source nature of unix made it obvious from the start that there were going to be lots of patches to things. The nature of collaboration meant that most changes were initially distributed as patches. And when I say patches I mean diffs, the kind of things that Larry Wall's patch utility takes as input, and he wrote that in 1985. Thus, patches were inherent to applications, not a special problem that manifested itself later. Additionaly, the unix philosophy involves making small functionally targeted applications "Do one thing and do it well", which means lots of bits got bundled together to make an application. This meant that the average Operating System would need tons of applications. With this in mind, people started creating tools to help manage lots of these packages of applications and their constant updates.

If we zoom forward to present day we see that this paid off. Debian has apt, FreeBSD has ports, RedHat has RPM, Gentoo has portage etc. All the 'derivatives' like OpenBSD and Ubuntu have them too. These tools all allow you to manage keeping your system up to date, and they do it well. I could replicate Microsoft's current WSUS infrastructure several years ago by setting up a central repository which usually just requires some default networking protocol like http or ftp, and pointing the 'agents' installed by default on every one of these distributions for the last couple of years at it. If you want to start getting fancy, like say adding internal organisational signatures, well slap your own GPG signature on the patches and use scp with a for loop in a bash script to update the GPG signature on every agent in your organisation. I don't need to pay Shavlik $1million for that. All of the other difficult stuff like dependency checking, binary diffs and efficient deployment is done already and doesn't require any modification of the base system.

In contract, Microsoft only realised that security patches are a special case later on and has rushed to try and provide the same level of efficiency as the unix world. Also, Microsoft applications are more often bulky behemoths that try and do everything. They haven't been subject to an evolution which made available a high quality set of building blocks. Even where applications with a lot of functionality are relevant (like OpenOffice or Evolution) they still build nicely on other libraries and toolkits, preventing the kind of situation we saw in Microsoft GDI+ JPEG vulnerability where nearly every application bundled its own version of gdiplus.dll. Microsoft's avisory on Microsoft products only, mentioned over 50 products and included links to 30 seperate updates. Not to mention the seperate patches required for third party applications. Even worse, you needed a third party tool to try and figure out which applications included the vulnerable library, because Microsoft's software inventory just records installed software, not dependencies. An equivalent query on my Debian machine would take a few seconds by running apt-cache showpkg <pkgname> and not requiring any third-part tools, because there is no such thing, the entire OS is made of of third-part tools.

But the technical aspects are the easy bits. They take up precious little space (unfortunately) in my MSc thesis on this subject. The hard parts are the processes around how all of this should work. There is a whole bunch an organisation can do, you can read about that when I finish my thesis, but that is common to all vendors. The difference is in what the vendor does. Now try and imagine this world. A critical vulnerability is announced in the latest version of Adobe Acrobat reader, and since everyone has that installed you want to patch it quickly. The problem is Adobe only released the patch for the latest version and your organisation's standard image has the previous version installed. So Microsoft kindly takes the fix from the latest version and backports it to the older versions and within a couple of hours is distributing it for 36 architectures. Now your organisation only has to go through minimal regression testing on the patch because there is no new functionality, only a small security fix. Better still, because it was distributed by the vendor you can use the standard deployment tools instead of figuring out how to fit Adobe's own deployment tool behind the corporate firewall. Now imagine this was true, not just for Acrobat Reader, but for *every* patch released by a *huge* number of vendors.

Well, that would be nice, but would literally be impossible in a Microsoft world. The code sharing agreements, the extra work required by Microsoft techies it would be a nightmare. But not in our happy unix open source world. This is exactly what happens in the case of FreeBSD and Debian (and possibly others, I am just not that familiar with them). Better still, this is free! You don't have to pay Patchlink a newborn every month, now you can put that company daycare centre to good use. This isn't only available to big corporates who employ Linus Torvals himself, but is available to any end user. Ubuntu linux for example follows the exact same model. Now you can provide the kind of patch management quality many Microsoft based corporates are still trying to figure out who they can pay to get to your mother and grandad.

The thing that scares me is that the average Windows administrator seems to be absolutely clueless about any of this. Microsoft has managed to sell them a world where nobody else innovates, or if they do they innovate for hairy geeks and not serious business people. So instead of any serious debate about the various methods of patching we are presented with, you get people posting news stories like this.

Aaah, rants do provide for a nice break from academic writing.

Blue Lane Technology's Patch Point

nospam@example.com (Dominic White) — Thu, 19 Jan 2006 15:04:27 +0200

Blue Lane seems to be getting more attention. I found this post through SA's own Dimension Data's blog. I once wrote about Patch Point when a reader asked for my opinion on it. At the time their site had very little information and I suspected that the product was snake oil. Since then more information has been added, and the post at Rational Security cleared up my understanding.

I am now prepared to roll back my claim that it is snake oil, but it still looks like a stupid idea. The basic idea is that the 'virtual patch' will modify network traffic in the same was that the vendor's patch will. Thus if the vulnerability involves sending over large UDP packets, the inline patch will truncate them, or if it involves a SQL injection, the inline patch will strip the offending SQL.

This bring me to several point, which I will summarise:

Why is stripping a malicious request better than blocking it?
How is this different from an IPS?

Stripping

The basic idea behind current IPS involve two parts, detection and prevention. The first is the hard part, actually detecting the attack, sometimes developing a signature is easy and sometimes not. Take the recent WMF exploit as an example, it requires reading a significant amount of data before a proper match can be made (in the worst case the IDS will have to buffer the whole image). In addition, the many variations on the WMF that can be made (for example HD Moore's gzip encoding tricks and header padding) mean multiple signatures need to be developed. Virtual Patching seems to ignore this, it focuses on a white list approach, which is usually much better.

What a virtual patch would do in this case is strip out the SetAbortProc() code of the WMF. But then the same problem occurs and all benefits of whitelisting are lost. Now you have the same problem of detecting a malicious WMF, with the addition of trying to salvage some useful WMF from it, which has a mess of problems with it. It is interesting to not that this isn't 'virtual patching' as the markeing team would have you believe, there is no way a network device can actually modify the GDI+ code without an actual patch, this is stripping the malicious request.

Then, given the other DoS conditions found in the GDI library for WMF, an IPS that just blocked the traffic would have kept you safe from that, whereas the 'virtual patch' wouldn't. This isn't some magic trick, it is just that the idea of blocking malicious traffic works on the assumption that malicious traffic is evil and none of it can be trusted. The, example used in the rational security post is that of a $50 million wire-transfer. When the hell would a legitimate $50 million wire-transfer contain a malicious request? Even assuming it does, once you know the transfer has been compromised, you don't try and fix it and continue, especially in this kind of high risk situation. You have some kind of intelligent failover which hopefully involves telling the user they have been compromised and to try again from a secure terminal.

The other example given by rational security is an active FTP session, where a legitimate user then goes crazy and tries to delete everything. An awkward example, but let's run with it. An IPS can already handle this, it can deny the delete request and let the rest of the session run amok. You will have to jump through some TCP hoops if you want this behaviour, but it can be done. This is a DUMB way of doing things however. If you want to deny all users from using the delete request then disable it in your FTP server, don't try and filter it out of the network traffic, this needs to be conducted in the application layer.

In summary, stripping a known malicious request of it's know bad parts is a bit like letting known criminals into your bank vault after taking their gun. It has the same complexity and problems in detection, but has a silly solution once detected.

Is this an IPS wearing a marketing frock?

It depends on your taste in frocks, but from where I am standing it seems this is an IPS with a slight difference. It suffers from all the same difficulties an IPS suffers in detecting attacks and the difference will more likely expose you to more risks rather than less.

Q: But, what about defense in depth? Why not use Patchpoint AND an IPS?

A: Because it would be redundant. If the IPS blocks the request before Patchpoint, then Patchpoint doesn't do any work, it is dumps it afterwards then Patchpoint wasted its time, if Patchpoint strips the malicious part and then the IPS lets it through, then you are exposing yourself to potentially malicious traffic for which a signature hasn't been developed yet.

Oh wait, there is one potential benefit of Patchpoint: the employees could produce high quality detection signatures. In this case, it should be compared to something like snort to see if their few employees do a better job than the snort community, I mean you could always turn off the inline patching and have it block malicious traffic. Given the price of the thing (starting at $30 500) it better be MUCH better than (the free) snort. If you are going to pay money, pay it to Tenable.

To quote Thomas Ptacek's Second Rule Of Security Marketing

If your inline network security device claims to provide "virtual patching", the box must use the actual binary patch from [the vendor] to do it.

UPDATE: Chris of RationalSecurity has responded, I don't think he adequately dealt with my points. Although he is being a good sport about it.

Oracle Patch Release - Less is More

nospam@example.com (Dominic White) — Wed, 18 Jan 2006 07:01:46 +0200

Is it just me or did Oracle release over 100 patches in their latest critical patch update (CPU, good thing they chose an acronym not used for anything else in computing). They claim that some vulnerabilities affect multiple products and their 'risk matricies' list the vulnerability for each product, which I assume would also need to be patched for each product.

They can't seem to get it right. They either release too few patches, ineffective patches, no patches or now, too many patches. Then again Pete Finnigan who knows his Oracle seems happy enough, so maybe I am missing something.

Oracle is following its usual 'partial disclosure' policy. Although several of the vulns are being researched and fully disclosed at

Good luck testing that sucker.

LyX Table Frustration

nospam@example.com (Dominic White) — Sun, 15 Jan 2006 03:27:49 +0200

Creating tables in LyX is a nightmare. This LyX wiki entry describes how to perform the 'missing' table manipulation functions.

Patch Time Graphs

nospam@example.com (Dominic White) — Sat, 14 Jan 2006 13:43:04 +0200

After fiddling with Brian Kreb's work yesterday, (available at SecurityFix) I decided to take it a step further and draw some pretty graphs. Here the patches were sorted into chronological order based on the date of the original report. It is interesting to note that Microsoft patches vulnerabilities reported at the end of the year faster than they do those reported at the beginning. In the graphs, blue lines are full disclosure vulnerabilities and the orange are responsible disclosure vulnerabilities. The full disclosure graph also shows the large improvement in patch times in those cases. I used a 5-point rolling average for the trend curves. It is interesting to note the cyclical nature of the Patch Times on the summary graph. There aren't just random spikes and troughs there are usually other highs and lows building up to them. It would be nice to know what projects were on at Microsoft that may lead to the general increase in patch times over that period, alternatively it could be the nature of the vulnerabilities. Any ideas?

Also in the last three years, Microsoft has:

Released 99 critical patches
Taken an average of 120 days to release a patch
Taken an average of 62 days to release patches for full disclosure vulnerabilities

The original spreadsheet is available in:

I changed the day calculations so that they will work in Excel, however Excel is unable to display the graphs correctly and just shows two sets of bars instead of bars and a trend line, so I recommend either the OpenOffice version or the HTML.

As an aside, what are the correct terms for the two types of disclosure. Responsible disclosure is a rather morally laden term, and calling the alternative irresponsible or non-responsible seems silly. I am using 'full disclosure' in this entry, but it seems wrong.

Microsoft Patch Speed Inconsistencies

nospam@example.com (Dominic White) — Fri, 13 Jan 2006 15:13:42 +0200

While going over the research on Microsoft's time to patch produced by Brian Krebs at SecurityFix, I noticed a few things which didn't add up. His calculations for the number of days from internal or full disclosure until patch release appeared wrong. On double checking it seems they were. The calculations for 2005 were particularly bad with a total of 118 days going missing or being added. There are many off by one errors and in one case the disclosure date was listed after the patch release date, once the year was changed from 2003 to 2002 it made sense. For both 2003 and 2004 the number of patches were counted incorrectly! Given that the information was vetted by Stephen Toulouse of Microsoft, it is strange they they both missed this. The other possibility is that I have missed something, anyone care to double check my calculations? Brian has since seen this post and linked to it.

A spreadsheet is available with my calculations next to Krebs. In my corrected days column I have italicized and centered the days where my results and his disagree. ~~I used Open Office's DAYS() function~~ I just do a normal subtraction to calculate the difference in the days.

While the errors were sometimes quite large, the average calculations are not badly affected as the days were sometimes higher, and othertimes lower than they should be. The dates are still hugely useful, and all sorts of interesting information can be derived from them (eg1, eg2), it would be nice to have the same info for other vendors. Thus, the new summary is:

	2003	2004	2005
Number of Critical Patches	34	28	37
Ave. Days from Report to Patch	90.7	136	134
Ave. Days from Disclosure to Patch	73.6	55	46

UPDATE: added link to SecurityFix's follow-up post and Dan Geer's work