Managed Mayhem

Lately I’ve found myself in that weird spot where I’m both applying for jobs and seeking contract work at the same time. It is inevitable that one of two things is going to wind up happening during the process:

• I land some contract work, get busy… and then find a job.
• I land a job, get busy… and then get swamped with contract work that I either take on myself or need to delegate.

Don’t take me as being ungrateful for either … I’ve been without ‘real’ work for approaching a year now, and so either way the income would be a pleasant change of pace.

If you’ve ever been in the job market, one of the things that I’m sure you’ve noticed, is that there are many employers who require a degree before they will even consider interviewing you. I mean, I get it. Hiring managers and HR folks want to be sure that the candidate has some depth to them, and they are assuming that a college degree provides that depth.

What follows is a history of how I have obtained that depth without a college education. Skip to the bottom if you want to avoid falling asleep.

I got my start in computers at the age of 9. My dad bought a Texas Instruments Ti994a. I mostly played a game called ‘Parsec’ … which worked great with the speech synthesizer that came with the computer. By the time I was 11, I was programming in Atari Basic on an Atari 600xl, saving my creations to a cassette. Then Apple Basic on an Apple II+ my mom got from her work when they upgraded. I got my first job programming an inventory control system for a local drug store when I was 12. Then Logo (a drawing program) on an Apple IIe, WildCat BBS on a Commodore Amiga, GW Basic on a PC-AT 8088 when I was 14, Apple Pascal on an Apple IIGS (thank you Woz!) when I was 15. In high school I helped to teach computer programming in my sophomore year as a teacher’s assistant. I also built a student matching program that took surveys given to incoming freshmen and outgoing seniors and matched them similar to big brother / big sister.

I learned Borland C and C++ (C with classes) on a dual 486 I bought from an ISP I worked at when I was 18. Then I learned Html and Livescript/Mocha/Javascript and a bit of Java on that same machine. While working for that ISP, I also learned NCSA HTTPd (apache), Unix (FreeBSD), DNS, Sendmail, QMail, Pine, Kermit, Lynx, VT Terminals, dialup networking, TCP/IP… etc…), and helped them to support their dialup customer base.

This job was my BS in MIS and CompSci.

I went to Penn State for 1 year between ‘94 and ‘95. I always joke about how I learned ‘Shakespeare and Beer’. Once I got into my major (Computer Science), after two years of general studies, I would have been learning Fortran, Forth, Cobol, and re-learning C. Meanwhile, the internet was quickly changing the world and the way we live and communicate in it, Linus Torvalds had just gotten started with Linux, C++ was gaining in popularity, and the web was growing. So I quit college.

That same year, I hooked up with some folks at NetDay, helping rural communities span the digital divide by bringing together the resources and people needed to get them online with broadband. We helped to connect over 100 schools in my area. I also worked as a volunteer for LibertyNet just before it was bought out by Region Online.

My first ‘real’ job as a software developer was working for a company in Hartford, CT called ‘Insure.com’. They were an insurance portal that made their money off advertising, and they hired me to work on their CGI based banner advertising system, So I learned enough Perl to be dangerous while hacking through thousands of pages worth in Html and server side includes. While working there, I also took on a volunteer position with a small company named Lycos. At the time, they were still a part of Carnegie Melon, and they had an online chat community called ‘LycosChat’. I was a ‘Volunteer Coordinator’ for them for over a year before I moved to Tennessee. I love Tennessee, but In hindsight… I probably should have moved to California instead.

When I moved to Murfreesboro, TN in 1998 … My goal was to go to MTSU for Recording Industry Management. I was unable to meet the residency requirements in my first year here, so I started working. My first career related position in Murfreesboro was working for a local commercial real estate firm helping them with getting their website up. I started a small web design business out of my apartment with a friend of mine, and wound up getting enough work to justify not going back to school. I started helping a friend of mine with a small dialup ISP he had started, and through that relationship I also landed a good sized contract with a company that wanted to manage computer parts inventory online. I hired a contractor / friend to help build it. Allaire had just released ColdFusion version 4, and my friend was actively developing applications using it. It was my first introduction to ColdFusion, and he would come over to my apartment and work on my couch on his laptop. At one point he got a job, and was unable to finish the project… but had left behind Ben Forta’s ‘Web Application Construction Kit’ … and so that’s how I learned Cold Fusion in order to finish up the project. Trial by fire. This was also when I learned Fusebox.

At the time, I had been hosting my clients with a company called ‘Ichthus’. They were ok, but I felt that I needed better margins, and so I decided that I would get into the hosting business. I had met a guy in Maryland during a trip I took out there… I felt drawn to this small computer shop / ISP that I saw while I was eating at a Burger King across the street. Nathan Whitaker and I hit it off immediately, and it wasn’t long before I had a server running NT4 on his T1 line in his shop. My partner in Murfreesboro had moved to California, and had given up his share in the business … so at that point I was back to freelancing on my own, and I learned NT4, IIS, Microsoft DNS, Imail, and Sql Server 7. I continued to contract web and flash designers, and made a decent living doing it.

At some point in ‘98-99 … I was introduced to a group of people in Nashville who were starting a small collective of designers called The615. I was technically one of the founding members (managed to get my picture taken with the rest of them at the first meeting) … but did not really become involved until a year or two later, and even then only as a support for the email lists and occasional contributor of long emails.

In ‘99, I started another company called ‘Dog Ear Digital’ with some friends… merging all of my existing clients into it. Two of the four initial partners (myself and one other guy) left to pursue school or other careers. Me and the remaining partner eventually decided to go our own way as well, each of us taking the customers we brought into the relationship with us when we left.

In 2001, Netfinity3 Inc. was born while I worked full time for Nashville Electric Service (Java, ColdFusion, and AS400!), and then Dave Ramsey (Cold Fusion Software Engineer). I ran Netfinity3 with my wife for 7 years…coming on board full time in 2003. I learned delegation, sales and marketing, Project management using SDLC, got better at Cold Fusion, Fusebox, and MSSQL (2000 and 2005), learned PHP, MySql, PostGres, Java, ASP Classic, .Net, Asterisk VOIP, Cpanel, Ensim, Plesk, ModernBill, Tomcat, Exchange, MDaemon, managed a mid-sized cisco, riverstone, windows 2000 and linux server hosting network in 3 different datacenters, managed an outsourced callcenter in Knoxville, a team of server administrators in Mumbai, and an outsourced networking, software development and web design team spread throughout the US, Canada, Romania, Ukraine, Belarus, and Israel. I learned a lot about customer service, accounting, and bootstrapping a startup business without debt or investment. I consider this my MBA and Masters in MIS, and I got paid for doing it rather than paying Harvard for it.

We sold Netfinity3 in 2007 when someone offered to buy it, and then I started working full time again. Since 2007, I’ve developed experience in XSLT, XQuery, XPath, Oracle, Ajax, Flex, Eclipse, Air, Ruby (a little bit), Python (basic understanding), and I continue to develop my skills as a Cold Fusion developer. I’ve worked for AOL, Just1Word, have dabbled a bit in real estate and a landscaping business, and have taken the chance with a couple of startup companies as well.

Some of my favorite business, leadership, and marketing authors are: Seth Godin, Malcom Gladwell, Guy Kawasaki, Thomas Friedman, Jim Collins, Michael Gerber, Marcus Buckingham, John Maxwell, Thomas Stanley, and Robert Kiyosaki. I consider myself well read and educated, despite only having sat in a university classroom for 1 year.

Hopefully none of this sounds like bragging… it was really intended to give some context of some of the history of my career over the last 17 years to show the depth that can be created without a college education. I’m obviously not against education… not at all! In fact, I learned Python at MIT!

My feeling is that you can get a sheepskin, and still remain a sheep… or you can just be a wolf with or without the sheepskin and be aggressive about continuing your education your entire life. I have consistently chosen to be a wolf. This choice has only hindered me in cases where I’ve applied for jobs that require a college degree, and has essentially challenged me to be more of an entrepreneur and to re-invent myself more often in order to stay marketable. My wife recently said ‘I’ve never known you to be without something to do’. I tend to keep busy doing something even when I’m not ‘working’. I think that’s a good thing.

Maybe this is a plea of sorts for hiring managers to look beyond the requirement of a degree. There might be a few of us wolves out there.

A few months ago, I wrote about Quantum Cryptography and Quantum Key Distribution using the ‘Observer Effect’:

http://www.managedmayhem.com/2009/05/20/quantum-cryptography-quantum-key-distribution-qkd/

I got to thinking today about how possible it would be to use this same concept of quantum state entanglement in order to communicate across long distances. My thought was, if QKD exists, why not use the entangled photons… one in a transmitter, the other in a reciever… and change the state rapidly in an ‘on / off’ pattern similar to binary? This would effectively change the state of the receiver photon, which could be observed (syn / ack) and then translated into a more classical form.

As it turns out, they are already working on it:

http://www.universetoday.com/2008/03/17/a-step-toward-quantum-communications-with-space/
http://arxiv.org/abs/0803.1871
http://www.niac.usra.edu/studies/974Morgan.html
http://arxiv.org/abs/quant-ph/0305105v1

I’d love to see this stuff in operation 20-30 years from now.

I have dealt with every single one of these types of clients before, and have been amazed with how frequently people will expect creative industry folks to do things that they would not ask their mechanic or doctor to do. Why is there a different level of expectation and total lack of respect for the time and talent of designers and developers vs. other professions? Why does it always seem to be a negotiation whenever giving prices? Would you expect your doctor to work on spec?

For more information on spec work, and why it is bad for the creative industry as a whole, go here:

http://www.no-spec.com/

I’ve been reading up on security options for a financial services client of mine… and came across Quantum Key Distribution. Great stuff if you’re needing a secured point to point fiber optic network. The principle behind this stuff is that it uses quantum mechanics instead of the ‘mathematical difficulty’ normally associated with other classical cryptographic methods. Two devices (one on each side of the conversation) transfer photons as shared keys. One example of how QKD uses quantum mechanics is the idea that any quantum object that is observed (measured) changes, and so therefore we are able to detect when a photon ‘key’ is observed while in transit. Hack attempts can be detected (and prevented) in this manner because if the shared key is observed, the change makes it useless. This is important because using classical methods of cryptography, there is not really any way to determine if someone is ‘listening’, capturing packets for decryption. Another principle in use is ‘Quantum Entanglement’, which essentially ‘connects’ two quantum objects (in this case photons). This technology is currently in use, but primarily by governments and large corporations like banks.

http://en.wikipedia.org/wiki/Quantum_cryptography

This is a company in New York that sells QKD devices:

http://www.magiqtech.com/MagiQ/Products.html

I’d love to play with a couple of them.

From the browsershots.org website:

“Browsershots makes screenshots of your web design in different browsers. It is a free open-source online service created by Johann C. Rocholl. When you submit your web address, it will be added to the job queue. A number of distributed computers will open your website in their browser. Then they will make screenshots and upload them to the central server here.”

4 Platforms (Linux, Windows XP, Mac, Debian).
Almost 80 browser / version / OS combinations.

What I love most about the site… is not just what it can do, though it is pretty fantastic. What I really love is it’s use of distributed computing. The ‘distributed computers’ that they are using are actually computers that have been volunteered for use by folks on the web. Browsershots.org makes available a downloadable installation (written in Python) that is responsible for receiving screenshot requests and then fulfilling those requests, uploading them to the mother ship after it is completed. Genius.

Since the early-90’s and the RSA Challenge, I’ve been fascinated by the concept of distributed computing, particularly volunteer / open source distributed computing that makes use of available cycles / resources on people’s otherwise unused computers. Seti@Home uses this method to ‘listen’ to the results of mountains of data coming from space telescopes around the globe in a search for extra terrestrial life. FightAIDS@Home helps to identify candidate drugs that have the right shape and chemical characteristics to block HIV protease. Many of these projects have latched onto IBM’s ‘World Community Grid‘ program, running on the ‘BOINC‘ platform at about 2 Petaflops, and over 500,000 volunteered computers.

Service Oriented Architecture (SOA) with web services provides a great method for accomplishing some of these things. Now I want to build a distributed computing application.

Factory methods are responsible for manufacturing objects.

From ‘Gang of Four’:

‘Frameworks use abstract classes to define and maintain relationships between objects. A framework is often responsible for creating these objects as well’

Coldspring as a framework has an excellent implementation of the Factory Method pattern.

Example:

Let’s say I am building an application that has ‘CDs’ as objects.

If I am using a Service / Gateway / DAO pattern in my application… (I know that many folks rail against the 5:1 approach, but let’s just say for sake of argument that transfer is not a good fit for this application.), I have quite a bit of code to write just to get to a point where all of my required objects are instantiated:

<cfset datasource="library">
 
<cfset cdBean = createObject("component", "extensions.com.jimrising.cd.cd").init(title: "Joshua Tree", artist: "U2")>
<cfset cdDAO = createObject("component", "extensions.com.jimrising.cd.cdDAO").init(dsn: #datasource#)>
<cfset cdGateway = createObject("component", "extensions.com.jimrising.cd.cdGateway").init(dsn: #datasource#)>
<cfset cdService = createObject("component", "extensions.com.jimrising.cd.cdService").init(cdDAO: cdDAO, cdGateway: cdGateway)>

Note that the ‘cdService’ depends on the injection of the cdGateway and cdDAO. Coldspring handles this dependency injection nicely as well.

In order for me to create a new ‘cd’, and then save it to the database:

<cfset cdService.save(cdBean)>

The issue is one of persistence. In order for me to use these objects throughout my application, without the Factory Method, I would need to instantiate them each time I needed them. Using the Factory Method and a little setup within Coldspring, these objects persist and I’m able to access them through the factory, without needing to know all of the dependencies or subclasses:

<cfset application.serviceFactory.cdBean.init(title: "Joshua Tree", artist: "U2")>
<cfset application.serviceFactory.cdService.save(cdBean)>

Note that the DAO, Service, and Bean have already been instantiated, and so saving a new ‘cd’ object to the database (from anywhere within your application) is a simple matter of building the bean and saving it through the service within the serviceFactory.

Here is what the XML configuration (services.xml) looks like for ColdSpring:

<beans>
    <bean id="cdGateway" class="extensions.com.jimrising.cd.cdGateway">
        <constructor-arg name="dsn"><value>${dsn}</value></constructor-arg>
    </bean>
    <bean id="cdDAO" class="extensions.com.jimrising.cd.cdDAO"> 
        <constructor-arg name="dsn"><value>${dsn}</value></constructor-arg> 
       </bean>
    <bean id="cdService" class="extensions.com.jimrising.cd.cdService">
        <constructor-arg name="cdDAO"><ref bean="cdDAO" /></constructor-arg>
        <constructor-arg name="cdGateway"><ref bean="cdGateway" /></constructor-arg>
    </bean>
 
    <bean id="cdBean" class="extensions.com.jimrising.cd.cd" />
</beans>

And the simple one time instantiation of the serviceFactory within your application.cfc onRequestStart() method:

        <cfset application.defaultProperties.dsn = "myDsn"/>
        <cfset application.serviceFactory = createObject('component','coldspring.beans.DefaultXmlBeanFactory').init(structnew(),application.defaultProperties)/>
        <cfset application.serviceDefinitionLocation = expandPath('/config/services.xml')>
        <cfset application.serviceFactory.loadBeansFromXmlFile(application.serviceDefinitionLocation)/>

Waterfall Model
The Waterfall software development methodology is one of the most widely known and recognized methodologies. Originally designed for the manufacturing and construction industries, it is called ‘Waterfall’ because of the way that it’s phases flow downward, similar to an actual waterfall. It is best uesd for projects where the requirements are clearly stated and static, or where it helps to have a rigid management structure with up-front requirements and agreed timeline and budget. Many times the decision to use Waterfall over another method (such as Agile) is simply a matter of the personalities involved in the project, including the project sponsor. Using the Waterfall software development life cycle, the implementation of the system is preceded by requirements definition, analysis, design and development. One problem with the traditional waterfall method, is that it is impossible to perfect one phase alone before moving to the next phase. Another issue with waterfall in general is that the nature of design in any creative field (software development included) makes it difficult if not impossible to define all of the requirements up-front or even prior to completion. In a project where requirements are always changing, and new ideas are being formed as design proceeds creates a ‘moving target’ syndrome where the waterfall model just does not fit.

Sashimi Model
The sashimi model (so called because it features overlapping phases, like the overlapping fish of Japanese sashimi) was originated by Peter DeGrace. It is sometimes referred to as the “waterfall model with overlapping phases” or “the waterfall model with feedback”. Since phases in the sashimi model overlap, information of problem spots can be acted upon during phases that would typically, in the pure waterfall model, precede others. For example, since the design and implementation phases will overlap in the sashimi model, implementation problems may be discovered during the design and implementation phase of the development process.This iterative method helps alleviate many of the problems associated with the traditional philosophy of the waterfall model, but does not fully address the organic nature of some software projects. Theoretically, one thing that it is good at is managing your costs during the project by keeping your iteration between two phases at a time. Ideally you will not iterate backwards beyond the previous phase, as if you are already developing and find that you are iterating back to requirements… there was a (potentially expensive) mistake made during the ‘Design and Architecture’ phase. Because this modified format of the traditional waterfall method is one of the most common (It is the one I use when using waterfall), this is the one I will be focusing on within this article.

Requirements
One of the most important tasks in the development of software using the waterfall method is gathering and defining  the requirements for the project.  Software requirements analysis is a rather broad field of software engineering and project management, but simply put … at the end of a requirements phase within the waterfall method, all involved parties should have a basic understanding of what is going to be developed, and at this point the requirements are written as ‘prose’, and are not usually very technical in nature. In some cases (particularly with smaller projects), a budget and timeline estimate is also formed, though in my opinion this is best done after the Design and Architecture phase, when the majority of the costs are more accurately known. For this reason, I will normally split my billing into two major phases … the first includes the first two project phases, ‘Requirements’ and ‘Design and Architecture’, usually billed on retainer. The second major phase includes the remaining 3 waterfall phases, split into three major billing phases of ‘Development’, ‘Testing’, and ‘Implementation’.

Design and Architecture
When discussing this phase of the waterfall method with my clients, I compare it to the design and architecture of a construction project. Before you build a custom building, you need to go to an architect in order to have that building designed. When building software, it is very similar. Software developers who specialize in this are even called ‘Software Architects’. During the Design and Architecture phase of a project, a Software Architect is responsible for defining with the project sponsor the functional and / or technical definition of the project. At times, tools such as wireframes and/or storyboards are used in order to help the architect to communicate with the developers and the project sponsor. Other times, UML (Unified Modeling Language) is used… though this is generally only used to communicate with other developers, as most project sponsors are not going to understand it. I have even developed the user interface of a software project during this phase in order to help the communication process between our team and the project sponsor. Keep in mind that the Sashimi Model that we’re using is iterative, so we’re constantly going back and forth between this phase and the last. Until the prior phase is fulfilled by this one.

The tangible result of the Design and Architecture phase should be a solid plan that defines the platform that is going to be used, the flow of the application, a hardware specification showing what hardware will be used… etc… Other tangibles may include the aforementioned wireframes, storyboards, and sometimes even a prototype user interface design.

Development and Coding
Once we all know what we’re actually doing, now the fun part begins! Part of the reason why the initial two phases of the waterfall process are so important is that this phase is the most time consuming and most expensive. Getting something wrong at this phase will inevitably mean one of two things… rework, or an unhappy client. Even when rework is done, this creates a situation where the client’s project might be late, or if not hers… someone else’s! Again, since we are using Sashimi Waterfall, the ‘Development and Coding’ phase overlaps with the ‘Design and Architecture’ phase until this phase fulfills the requirements of the one prior to it.

Quality Assurance and Software Testing
One of the most obvious improvements to the waterfall model when using Sashimi is during the testing phase. Because of the iterative approach of Sashimi, testing occurs as part of the development process, and then again as part of the deployment process. Develop, Test, Develop, Test, Develop… until the testing phase fulfills the development phase, Test, Implement, Test, Implement… until the application is fully tested and successfully deployed. Testing will also sometimes include several of it’s own minor phases… sometimes called ‘Alpha’ or ‘Beta’, other times managed as versions of the project with a version 1.0 project being a final first release. There are also several different types of testing:

Unit Testing
Integration Testing
System Testing
Regression Testing
Functional Testing
Performance Testing
Load Testing
Compatibility Testing

All of these are outside of the scope of this article, but I might be convinced to write another one down the road about testing.

Implementation
This is the phase of the project where the developed software is installed (or deployment scripts are delivered), documentation is written or cleaned up (as documentation should be written as an ongoing part of the development process), and sometimes client training will occur. Implementation is the only phase that I will sometimes overlap backwards two phases (between Deployment and Development) since there are sometimes things that are caught between Implmentation and Testing phases that require additional development to resolve.

Maintenance and Support
After the project is released into the wild, bad things can happen. This is why it is important that continued maintenance and support is addressed within any software development process. With the needs of clients and technology itself changing constantly, this support becomes an evolving process… and is essential to making sure that the software continues to perform as expected. I will normally move this phase out of the waterfall, and treat it as a separate project entirely as the nature of supporting an existing project can be very different from building a new project.

————————————————————————————–

As a side note, within my own waterfall process (when I use it), I have modified the waterfall model into an easier to remember system called the 5d’s:

Dream (pre-Requirements)
Define (Requirements)
Design (Design & Architecture)
Develop (Development and Coding)
Deploy (Includes Testing and Implementation)

Again, I will usually separate the maintenance of the project from the project itself, preferring a separate process for handling ongoing support.

Hopefully this article was helpful to those who are learning about the management of software projects. I think that all of the available models have something that you are able to take away from them. I’m not really a purist with any of them, and try to fit bits from all of them when they make sense for the particular project I’m involved in. Despite some of it’s faults, waterfall has it’s place… and I think that the addition of overlapping phases within the Sashimi model improves it dramatically. I probably would not use it without this modification.

I’ve been developing web software for over a decade now. As a result, I’ve had the opportunity to use quite a few project management platforms. I’ve even homegrown a couple along the way. The last several months, now that I am back to freelancing again, I have started using openGoo. The openGoo application is based on ActiveCollab’s last open source version, and it is a significant improvement both in functionality and form. It uses the EXT javascript library, which is a fantastic javascript UI, though it is a bit slower than others. Current stable release as of this writing is 1.3.1, and I’ve used it every day for the last month and a half with only very minor issues… which are expected in any open source project of this size. All of your projects are organized into ‘Workspaces’, with the ability to organize sub-workspaces as well. Security is fairly granular, allowing for user / group permissions at the workspace level. OpenGoo has the following modules included:

Notes
I use this primarily to keep track of workspace specific notes like login information, IP addresses… stuff like that.

Email
I have not used this yet because it is still in beta, but it enables you to check your email from within openGoo as a client. I can’t wait until they get all the kinks worked out!

Contacts
Pretty handy for keeping track of the folks who are involved in projects, though it would be nice if it integrated with existing CRM or contact systems. I use SugarCRM quite a bit, and so having to double enter contact information is a pain. You can import contacts and companies from a .csv file.

Calendar
openGoo comes with a very good calendar that integrates across all of the projects that you’re involved in so that you can see at a month / week / day view what you have going on. Really snazzy.

Documents
I use this in order to keep track of files and documents specific to the workspace / projects I’m working on. If a designer sends me a .psd file to slice up, I’ll put the original .psd file here. Any requirements documents will also go here. Content related to websites, customer agreements, etc…

Tasks
Probably the most used module within openGoo, you are able to assign tasks to any user, attach time to tasks, start date, due date, priority, milestones, and even tags. The task dashboard allows you to see what is current / past due at a glance, for all workspaces, all tasks, all users. There are also several sort options.

Weblinks
I don’t use this very often, but it is basically just a repository for links related to the workspace / project.

Time
Time reporting within openGoo needs some work. It is actually a bit buggy, but the way that time is assigned to tasks is actually very smart. I am looking forward to what they do to resolve the issues with the reporting functionality. For now, I export all of our time out of mySql using a small php report I wrote.

One of the things that I really enjoy about openGoo is that they have a very fast development cycle. I started messing with it at version 1.0 about 3 months ago, and they are already on 1.4.0 rc2. I’m amazed with how quickly they have been moving through releases. The community is a little weak / inexperienced, but the staff who are working on the project are very good about answering questions in the forums. This is huge for an open source project that is used for commercial purposes, especially for something as important as project management.

I’d like to see how they handle the billing capability within it though… right now I’m not so happy with the fact that it does not allow you to set per workspace billing rates or per service rates either. All billing rates are based on the user performing the work. This also does not take into account your actual cost as a development firm. The way that they handle accounting within openGoo is not the greatest, but I think that they will improve upon it.

All in all, openGoo is one of the best collaborative platforms I’ve used in a long time.