Lean Software Development according to Poppendieck

To create a Lead process, waste should be eliminated.

Taking this approach to software development (from requirements to deployments) helps explain many of the project failures I have saw or heard about.


In software development, the following are considered waste:

1. Partially Done Work
This is hard to first see, but a module that 'almost' works (but then abandoned for months) has very little value. So is a module that is so-called complete, but is untested, and therefore the bug-fixing work is not done.
This is similar to 'inventory' in classic lean; there is a high cost in having partially-done software (the knowledge gets obsolete, APIs and other systems shift, etc) which is not obvious to management.
Also, if it isn't deployed now, there is a chance that it would never be deployed.

2. Extra process
Clearly, stacks of paper have no positive impact except whatever benefit they did to the shipping code.

3. Extra features
Any additional line of code, and every additional feature, adds system complexity. Fixing a bug would now cost more; adding other (important) features would take longer. Carrying 500 pounds of dead weight in your car is only going to slow you down, and so are deadweight features

4. Task Switching
Every time a developer has to perform multiple tasks, they all take longer, and focus is lost.

5. Waiting
Having to wait (for approval, feedback, etc) slows the project down and wastes time

6. Motion
Especially in document handoffs, when every time a document is passed to the next person, knowledge is lost.

7. Defects
Bugs slow the process down since they require fixing, testing, and can cause other types of slowdown (such as causing developers to lose focus on their new features). Buggy code takes longer to complete.

Lean Software Development

Wikipedia states that "Lean Software Development is a translation of lean manufacturing principles and practices to the software development domain."

Lean software development, to me, is best viewed as both the philosophical underpinning of Agile, and as the extension of a similar approach to the entire software 'production' chain, from conception, through approval, to deployment.

The main points that a developer or a program manager would find helpful are:
* System: Optimize the system, not each part separately
* Value: understand what creates value for your customer, and how your organization creates value
* Flow: Maximizing speed minimizes wastes
This one is hard to see at first. But delivering a functioning solution earlier is critical since:
- You are producing the solution since it has value to the customer. The sooner the customer uses a functioning system, the more time he gets the benefit for, so the higher his total benefit is. If the core systems would save the customer $100k a month, delivering a core system 3 months later just cost $300k
- The time it takes to initiate and approve a project counts as cost, too. That stack of 'waiting for approval' projects on the VP's desk is costing the organization, the same way that the queue of people waiting for service in the grocery store costs you time.
- The more 'outstanding' projects there are, the more multitasking people need to do. Multitasking is expensive and inefficient.
- The more time passes, the less knowledge is fresh. Knowledge is what software development is all about.
* Pull: Deliver value based on customer (or next-in-production-chain) needs
And the perfect bad example is when the test organization is months behind the developers in testing. The dev team might be code-complete, but the product won't be shipping for six more months...


Special thanks to Alan Shalloway and his Lean presentation. The errors, of course, are all mine.

12 Basic Principles of Design For Perfomance

Goals and measurement:

1. Set performance goals
2. Set a performance 'budget' (e.g. 300ms for update etc)
3. Define performance testcases

Design:

4. Understand resources - disk, memory, network bandwidth, database connections, and local data protected by locks
5. Cut the deadwood - especially when reusing existing services and components, make sure your code doesn't do more (create more results, processes more data) than is needed.
6. Batch work - this will minimize roundtrips and enable efficiency gains from accessing resources only once.
7. Pool shared resources - such as database connections, TCP connections, or anything which has setup and teardown costs
8. Process independent tasks concurrently
9. Consider affinity - put resources near where they will be used; duplicate readonly resources (trade-off storage/memory for access time)
10. Use caching; design appropriate cache policy
11. Consider the efficiency of your algorithms; prototype algorithms early
12. Identify bottlenecks; there is always one more bottleneck

Programming languages popularity trends

This information is based on Google Trends (http://www.google.com/trends), which compares number of queries for a keyword
comparing the keywords C# and Java, Java queries are more popular by trend downwards, while C# queries trend upwards. Similar results come from comparing the keywords "c# source" and "java source", "c# programming" and "java programming"

Programming languages popularity

Trying to measure programming languages popularity is hard.
In an earlier post, I measured the number of the availability of jobs based on dice.com and similar places. Another interesting article is here. Both analysis supprt the view that C# and .NET jobs constitute the leading section of the job market, and more non-scientific research leads me to believe that .NET constitutes a majority of the NEW (as opposed to legacy, or code maintnance) programming jobs.

What do you think? I am looking for feedback, or your own experience.

telnet rules

A fact often missed by people too-used to web sites is that most services on the internet have underlying protocols. For example, if I want to do internic search on a domain name, I could google for a service that does just that; but I can just as easily get the information myself from port 43 of internic.net

Instructions:
telnet internic.net 43
type: =blogger.com
and you get (truncated):
Domain Name: BLOGGER.COM
Registrar: EMARKMONITOR IC. DBA MARKMONITOR
Whois Server: whois.markmonitor.com
Referral URL: http://www.markmonitor.com
Name Server: NS2.GOOGLE.COM
Name Server: NS1.GOOGLE.COM
Name Server: NS3.GOOGLE.COM
Name Server: NS4.GOOGLE.COM
Status: REGISTRAR LOCK
Updated Date: 22-jul-2005
Creation Date: 22-jun-1999
Expiration Date: 22-jun-2011

now, wasn't this easy?

.NET jobs exceed Java jobs

With the release of Visual Studio 2005 and the growing popularity and maturity of .NET, it seems more and more businesses are looking for .NET developers, as opposed to Java developers. Java popularity also suffers from competition from other Open Source languages and platforms such as Perl and Ruby (on rails).

For example, a quick search [Jan/2/06] on dice.com for .NET for '.NET' finds 9444 positions; 'Java' returns 2635.
Note that a search for 'C#' only returns 797 positions; .NET is by far a stronger brand name than C#, and the '#' sign wreaks havoc with multiple search engines.



Similar nationwide searches in hotjobs.yahoo.com read
Java : 6126
.Net : 10871
C# : # not supported in yahoo search

XML is also becoming a strong requirement:
XML (hotjobs.yahoo): 3396
XML (Dice): 6949



Tags: jobs .net java software market

Mambo and the problems of Open Source

Recently, trouble's been brewing in project Mambo.
Mambo source was originally developed by a company named Miro which chose to license the source under GPL, but retained copyright.
Conflcit arose over who controls the future of the project, the volunteer developers or the copyright owners.
An excellent outlines the developers' POV, and the developers website is OpenSourceMatters.

From the company's POV, however, I would think GPL-ing the source led to undesirable consequences. In short, once the source was GPLed, they lost all control over the future of the project - since the developers can (and did) fork the source, and continue development in the new fork.

While many hail this as a victory to the Open Source movemenet, which proved its supremecy over the corporation who developed the code, I am not so sure. This proves to be an incentive against corporations Open-Sourcing projects and cooperating with the OSS movement. OSS 'purists' would prefer it be that way, but as corporations do add value to projects, the OSS has lost many potential allies.

Tags: opensource mambo

Windows Communication Foundation ("Indigo") channels explained

I'm writing a serie of blog posts explaining the WCF channel architecture and extensibility point. The first in the serie is posted, others soon to come.

PMP Certification

I recently acquired the PMP certification, and wanted to share some insights.

What is PMP certification?
The PMP (Project Management Professional) is the leading project management certification, issued by the PMI (project management institute)

Why did I choose to get PMP certified?
In the software industry, project management skills are an intrinsic part of the roles of individual developers, leads, development managers, and of course project/program managers. While I am not currently looking for a project manager role, as the classical project manager seems too far removed from technology, I think the skills would be useful. And I might change my mind...
.. and, of course, the PMP designation looks good on my resume.

How was the test?
Hard. It's a hard test, the questions are misleading, and a 4 hours a test is a looong test.
I scored overall 86%. It seems traditional to share the sections breakdown, so here goes:
Initiating: 88%
Planning: 91%
Executing: 90%
Controlling: 84%
Closing: 79% [makes sense, I ran out of time before my final review of 'closing']
Professional responsibility: 86%

How did you study?
I used the Rita book, the Kim book, Achieve PMP success, the examcram2 book, the PMBOK (from which I learned very little, the style was too dry for my taste) and additional online resources - especially useful was the table of the 39 precesses, their inputs, tools, and outputs.
I spent half my time taking tests, and half reading textbooks or my notes. I estimate I put in 40-50 hours over 2 months - there seemed to be too much information for cramming.

Tags: PMP certificattion

Cisco, Mike Lynn, and my door

Recently, there was much discussion (also see here ) about Mike Lynn's presentation exposing a a Cisco vulnerability.
Details are a bit fuzzy. It seems the patch was already made by Cisco.
In some ways, exposing security issues is beneficial for the industry, which would otherwise ignore them and never fix them - an industry needs watchdogs to force it to fix product defects, and the car industry is an excellent example. And of course, it is very human to want to (and have the right to) discuss your achievements - and for a security researcher, finding a security vulnerability is a major achievement.
However, I refuse to glorify such actions. If a neighbor noticed my house door is open, and updated a bulletin board in the middle of the town ("houses with unlocked doors can be found at...."), I would be unhappy; and I don't think I'll be loaning him the lown mower again.

I think there should be a way for security researchers to get the fame (and improved job opportunity, and better pay, and everything an achievement usually entails) they deserve for finding security issues; and security researchers should show more responsibility on their part.
It boils down to processes and money. If a security researcher spends (on average) months and very sophisticated skill set finding a security issue, and he'll inform the company and make no big deal about it, he's been cheated out of any rewards for his efforts - and both the community and the company got security testing and review for free.
That's an untenable situation; as long as researchers have an incentive to disclose security holes, they will; but I can't quite see that sueing researchers would create the right kind of incentive.
What would I like to see? a substential monetary reward & public recognition program for security researchers who find issues, complemented by a much longer wait time on disclosing the details of any security issue.

Searching Blogs - compared

I found this link on Mary Hodder's blog, which I found here. It's interesting to note that no major search engine (Google, Msn Search, or Yahoo) currently supports real-time or near-real-time searches of blogs.

Windows codename Longhorn is now known as Vista

Vista? See this article .

Intel on multi-core processing

this article found on Intel website outlines their view on multi-core processors.

I totally agree with first 'take-away':
"For software executives, the first priority is to make sure your applications effectively take advantage of parallel proccessing capabilities of the multicore processors"
and mostly agree with the second:
"For enterprise IT management, multicore capabilities present major opportunities to lower the cost of computing through server consolidation"

Intel predicts that by the end of '06, expected run rate of dual-core CPUs on the desktop would exceed 70%, and hit 85% on servers.

Those suggestions are compatible with my suggestions in my MSDN Magazine article on hyperthreading, only more so - while hyperthreading has shown a modest performance boost, multi-core shows a greater persformance boost. This strengthens the position that future software performance boosts will depend on being able to write scalable multi-threaded applications.

Microsoft Certification upcoming changes

According to this article and other sources, the current Microsoft certifications, such as MCSD, MCSE, an MCDBA, will be retired (or at least no longer offered) around September.

They will be replaced with 3 levels of certification:
Tier 1: Microsoft Certified Technology Specialist
Tier 2: Microsoft Certified IT Professional & Professional Developer
Recertification will be required to maintain status at this level.
Tier 3: Microsoft Certified Architect
A board-level certification that requires recertification.

Additional blog post on the subject is on here.

As I post, the Microsoft certification page does not specify that any certifications are being discontinued, so the official story is not available.

SHA-1 break paper available

Recently, a paper by Xiaoyun Wang et al describes how collusions can be found on the common hashing technique SHA-1.
"In this paper, we present new collusion search attack on SHA-1".

This cryptographic volnurability has some far reaching implications as to the security of encryption systems.
For example, since digital signatures normally signs the hash of a document, an attacker might be able to forge a signature on a Word .DOC file - by taking a legitimate document D and modifying it (by adding spaces, changing file format, etc) to have a hash signature of another document.

As most cryptographic issues, there may or may not be an immediate issue, and solutions will have to be provided by cryptographic libraries providers. But it is a fascinating read - assumptions we made out-of-hand just years ago keep getting proven wrong. There's a lesson in it somewhere, if only I could find it.
Tags: software cryptography

Wardriving illegal?

this CNN.com article discusses how a man was charged with stealing Wi-Fi signal. AFAIK, this is the first time anyone is prosecuted for that. But totally insecure Wi-Fi networks are still a big percentage of home WiFi networks - even tho' securing them (to some degree, at least - even by specifying acceptable MAC addresses) is nearly trivial.

Indigo patents not a barrier to interoperability

This article states:
Microsoft said its willingness to file patents on its planned Indigo Web-services technology will not affect the software’s ability to interoperate with other vendors’ software.
Interesting read, but not that much meat in the article.

WikiWiki

By now, most everyone know about wikipedia, but if you ever wondered what the word wiki means?
Hawaiian for informal or quick, the internal bus in the Honolulu airport Honolulu is called the WikiWiki:


Tags: wiki

AMD vs Intel

Mary Jo Foley writes on the AMD vs Intel antitrust case.
Great read, although I don't always agree with Mary's perspective on the industry.