Content feed Comments Feed

Managed Mayhem

More Than 99 Billion Cats Herded

Archive for May, 2009

Managing Threads in Coldfusion

Posted by Jim Rising On May - 6 - 2009

[this is actually an older article from my previous blog, just moving it to my mm blog] …

cf_appicon

I started looking at <cfthread> today, and realized that it really does not have any sort of management facility built into it. while you can view / manage threads from within the server monitor…

1) you can’t get a status for a thread except from the calling page, or from another thread that was called from that page. this presents a problem if you have long running threads, or if you have other parts of your application that rely on thread status.

2) if there is an error that was generated during a thread process, again … you can access this at the page level, or from another thread, but there really isn’t anything that will help you to manage this within your application logic, say if you wanted to fire a thread again later.

Matt and I discussed an approach that could work well for managing these two things that are missing natively in cfthread, and act as a good foundation for further thread management. Keep in mind that this article really only consists of some raw ideas with pseudocode at this point. I’ll get around to writing the actual components later, and i’m sure it will develop into something much different from the ideas on this page.

The concept is based on two components that manipulate a server struct and act as a wrapper for cfthread:

the server struct consists of the following:

server.instance.threadName
server.instance.initTimeStamp
server.instance.startTimeStamp
server.instance.endTimeStamp
server.instance.elapsedTime
server.instance.function (join, run, sleep, terminate)
server.instance.priority
server.instance.object
server.instance.method
server.instance.argList
server.instance.status
server.instance.error

threadManager.cfc
This is the thread manager responsible for creating new threads, cleaning up old threads, and contains the getters / setters for thread status and errors.

thread.cfc
This is the thread object that acts as a container for thread processes. It builds a thread using <cfthread>, and creates a structure in the server scope containing the name of the thread, timestamp when it was created, current thread status, and any errors that occur.

myThread = thread.new(run, object.method, argumentList);

what this would do is create a new ‘run thread’ using the object.method(), given a name | value list of arguments. it would return the name of the thread as a UUID, and would add the thread instance to the server.instance struct. note that we’re passing in a list of arguments and values not for thread.new, but for the object.method.

newThread.exec(myThread);

this would execute the thread created previously, updating server.instance.startTimeStamp and server.instance.status in the server.instance struct. if the thread runs into any try/catch … this would be updated to server.instance.error. once the thread is completed, server.instance.status would be updated, as well as server.instance.endTimeStamp. all we’re doing here is passing the myThread object into newThread.exec, and then wrapping the object.method(argumentList); within <cfthread>

Agile Software Development Process

Posted by Jim Rising On May - 6 - 2009

agiledevelopmentprocess1From Wikipedia:

“Agile software development refers to a group of software development methodologies based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams. The term was coined in the year 2001 when the Agile Manifesto was formulated.

Agile methods generally promote a project management process that encourages frequent inspection and adaptation, a leadership philosophy that encourages teamwork, self-organization and accountability, a set of engineering best practices that allow for rapid delivery of high-quality software, and a business approach that aligns development with customer needs and company goals. Conceptual foundations of this framework are found in modern approaches to operations management and analysis, such as lean manufacturing, soft systems methodology, speech act theory (network of conversations approach), and Six Sigma.”

Agile software development processes are built on the foundation of iterative development. Agile processes use feedback rather than planning as their primary control mechanism. This software process model recognizes that software, like all complex systems, evolves over a period of time.

The main principles of the Agile Software Development process are:

  • Capture and define requirements at a high level
  • User involvement is essential
  • The team must be allowed to make decisions
  • Develop small, incremental releases and iterate
  • Focus on frequent delivery of products
  • Requirements evolve but the timescale is fixed
  • Complete each feature before moving on to the next
  • Testing is integrated throughout the project lifecycle - test early and often
  • A collaborative & cooperative approach between all stakeholders is essential

Requirements
The first step in the Agile Software Development Process is to identify some high-level requirements as well as the scope of the release. This 30,000 foot view allows developers to quickly begin coding in order to find out what works even quicker. While the requirements developed within a Waterfall Software Development Process are considered ‘law’, requirements within an Agile process are more or less ’suggestions’, and are open to more conversation by the team during other phases. Because requirements are not set in stone, the Agile method is more adaptable to changes in requirements as the project grows.

Architecture & Design
The goal of the architecture and design phase is to try to identify an architecture that has a good chance of working. The architecture is often defined using free-form diagrams which explore the technical infrastructure, and the major business entities and their relationships. The design is derived in a modeling session, in which issues are explored, until the team is satisfied that they understand what needs to be delivered.

Development
The development phase uses an evolutionary method that is an iterative and incremental approach to software development. Instead of creating a comprehensive prerequisite such as a requirements specification, that you review and accept before creating a complete design model; the critical development piece evolves over time in an iterative manner. The system is delivered incrementally over time, in small modules that have immediate business value, rather than building and then delivering a system in a single “big bang” release. By focusing development on smaller modules, Agile projects are able to control costs despite the seeming lack of planning.

Test & Feedback
One of the key principles of the Agile Methodology is to conduct the testing of the software as it is being developed. The software development is test driven. The unit testing is achieved from the developer’s perspective and the acceptance testing is conducted from the customer’s perspective.

————————————————————————————-

One thing to note about the Agile process is that it usually includes a more structured Sashimi Waterfall method inside of each iteration. If you look at the diagram above, you should see what I mean.

Sashimi Waterfall Software Development Process

Posted by Jim Rising On May - 6 - 2009

developmentprocess

Waterfall Model
The Waterfall software development methodology is one of the most widely known and recognized methodologies. Originally designed for the manufacturing and construction industries, it is called ‘Waterfall’ because of the way that it’s phases flow downward, similar to an actual waterfall. It is best uesd for projects where the requirements are clearly stated and static, or where it helps to have a rigid management structure with up-front requirements and agreed timeline and budget. Many times the decision to use Waterfall over another method (such as Agile) is simply a matter of the personalities involved in the project, including the project sponsor. Using the Waterfall software development life cycle, the implementation of the system is preceded by requirements definition, analysis, design and development. One problem with the traditional waterfall method, is that it is impossible to perfect one phase alone before moving to the next phase. Another issue with waterfall in general is that the nature of design in any creative field (software development included) makes it difficult if not impossible to define all of the requirements up-front or even prior to completion. In a project where requirements are always changing, and new ideas are being formed as design proceeds creates a ‘moving target’ syndrome where the waterfall model just does not fit.

Sashimi Model
The sashimi model (so called because it features overlapping phases, like the overlapping fish of Japanese sashimi) was originated by Peter DeGrace. It is sometimes referred to as the “waterfall model with overlapping phases” or “the waterfall model with feedback”. Since phases in the sashimi model overlap, information of problem spots can be acted upon during phases that would typically, in the pure waterfall model, precede others. For example, since the design and implementation phases will overlap in the sashimi model, implementation problems may be discovered during the design and implementation phase of the development process.This iterative method helps alleviate many of the problems associated with the traditional philosophy of the waterfall model, but does not fully address the organic nature of some software projects. Theoretically, one thing that it is good at is managing your costs during the project by keeping your iteration between two phases at a time. Ideally you will not iterate backwards beyond the previous phase, as if you are already developing and find that you are iterating back to requirements… there was a (potentially expensive) mistake made during the ‘Design and Architecture’ phase. Because this modified format of the traditional waterfall method is one of the most common (It is the one I use when using waterfall), this is the one I will be focusing on within this article.

Requirements
One of the most important tasks in the development of software using the waterfall method is gathering and defining  the requirements for the project.  Software requirements analysis is a rather broad field of software engineering and project management, but simply put … at the end of a requirements phase within the waterfall method, all involved parties should have a basic understanding of what is going to be developed, and at this point the requirements are written as ‘prose’, and are not usually very technical in nature. In some cases (particularly with smaller projects), a budget and timeline estimate is also formed, though in my opinion this is best done after the Design and Architecture phase, when the majority of the costs are more accurately known. For this reason, I will normally split my billing into two major phases … the first includes the first two project phases, ‘Requirements’ and ‘Design and Architecture’, usually billed on retainer. The second major phase includes the remaining 3 waterfall phases, split into three major billing phases of ‘Development’, ‘Testing’, and ‘Implementation’.

Design and Architecture
When discussing this phase of the waterfall method with my clients, I compare it to the design and architecture of a construction project. Before you build a custom building, you need to go to an architect in order to have that building designed. When building software, it is very similar. Software developers who specialize in this are even called ‘Software Architects’. During the Design and Architecture phase of a project, a Software Architect is responsible for defining with the project sponsor the functional and / or technical definition of the project. At times, tools such as wireframes and/or storyboards are used in order to help the architect to communicate with the developers and the project sponsor. Other times, UML (Unified Modeling Language) is used… though this is generally only used to communicate with other developers, as most project sponsors are not going to understand it. I have even developed the user interface of a software project during this phase in order to help the communication process between our team and the project sponsor. Keep in mind that the Sashimi Model that we’re using is iterative, so we’re constantly going back and forth between this phase and the last. Until the prior phase is fulfilled by this one.

The tangible result of the Design and Architecture phase should be a solid plan that defines the platform that is going to be used, the flow of the application, a hardware specification showing what hardware will be used… etc… Other tangibles may include the aforementioned wireframes, storyboards, and sometimes even a prototype user interface design.

Development and Coding
Once we all know what we’re actually doing, now the fun part begins! Part of the reason why the initial two phases of the waterfall process are so important is that this phase is the most time consuming and most expensive. Getting something wrong at this phase will inevitably mean one of two things… rework, or an unhappy client. Even when rework is done, this creates a situation where the client’s project might be late, or if not hers… someone else’s! Again, since we are using Sashimi Waterfall, the ‘Development and Coding’ phase overlaps with the ‘Design and Architecture’ phase until this phase fulfills the requirements of the one prior to it.

Quality Assurance and Software Testing
One of the most obvious improvements to the waterfall model when using Sashimi is during the testing phase. Because of the iterative approach of Sashimi, testing occurs as part of the development process, and then again as part of the deployment process. Develop, Test, Develop, Test, Develop… until the testing phase fulfills the development phase, Test, Implement, Test, Implement… until the application is fully tested and successfully deployed. Testing will also sometimes include several of it’s own minor phases… sometimes called ‘Alpha’ or ‘Beta’, other times managed as versions of the project with a version 1.0 project being a final first release. There are also several different types of testing:

Unit Testing
Integration Testing
System Testing
Regression Testing
Functional Testing
Performance Testing
Load Testing
Compatibility Testing

All of these are outside of the scope of this article, but I might be convinced to write another one down the road about testing. :)

Implementation
This is the phase of the project where the developed software is installed (or deployment scripts are delivered), documentation is written or cleaned up (as documentation should be written as an ongoing part of the development process), and sometimes client training will occur. Implementation is the only phase that I will sometimes overlap backwards two phases (between Deployment and Development) since there are sometimes things that are caught between Implmentation and Testing phases that require additional development to resolve.

Maintenance and Support
After the project is released into the wild, bad things can happen. This is why it is important that continued maintenance and support is addressed within any software development process. With the needs of clients and technology itself changing constantly, this support becomes an evolving process… and is essential to making sure that the software continues to perform as expected. I will normally move this phase out of the waterfall, and treat it as a separate project entirely as the nature of supporting an existing project can be very different from building a new project.

————————————————————————————–

As a side note, within my own waterfall process (when I use it), I have modified the waterfall model into an easier to remember system called the 5d’s:

Dream (pre-Requirements)
Define (Requirements)
Design (Design & Architecture)
Develop (Development and Coding)
Deploy (Includes Testing and Implementation)

Again, I will usually separate the maintenance of the project from the project itself, preferring a separate process for handling ongoing support.

Hopefully this article was helpful to those who are learning about the management of software projects. I think that all of the available models have something that you are able to take away from them. I’m not really a purist with any of them, and try to fit bits from all of them when they make sense for the particular project I’m involved in. Despite some of it’s faults, waterfall has it’s place… and I think that the addition of overlapping phases within the Sashimi model improves it dramatically. I probably would not use it without this modification.

openGoo 1.3.1

Posted by Jim Rising On May - 6 - 2009

logo_gooI’ve been developing web software for over a decade now. As a result, I’ve had the opportunity to use quite a few project management platforms. I’ve even homegrown a couple along the way. The last several months, now that I am back to freelancing again, I have started using openGoo. The openGoo application is based on ActiveCollab’s last open source version, and it is a significant improvement both in functionality and form. It uses the EXT javascript library, which is a fantastic javascript UI, though it is a bit slower than others. Current stable release as of this writing is 1.3.1, and I’ve used it every day for the last month and a half with only very minor issues… which are expected in any open source project of this size. All of your projects are organized into ‘Workspaces’, with the ability to organize sub-workspaces as well. Security is fairly granular, allowing for user / group permissions at the workspace level. OpenGoo has the following modules included:

Notes
I use this primarily to keep track of workspace specific notes like login information, IP addresses… stuff like that.

Email
I have not used this yet because it is still in beta, but it enables you to check your email from within openGoo as a client. I can’t wait until they get all the kinks worked out!

Contacts
Pretty handy for keeping track of the folks who are involved in projects, though it would be nice if it integrated with existing CRM or contact systems. I use SugarCRM quite a bit, and so having to double enter contact information is a pain. You can import contacts and companies from a .csv file.

Calendar
openGoo comes with a very good calendar that integrates across all of the projects that you’re involved in so that you can see at a month / week / day view what you have going on. Really snazzy.

Documents
I use this in order to keep track of files and documents specific to the workspace / projects I’m working on. If a designer sends me a .psd file to slice up, I’ll put the original .psd file here. Any requirements documents will also go here. Content related to websites, customer agreements, etc…

Tasks
Probably the most used module within openGoo, you are able to assign tasks to any user, attach time to tasks, start date, due date, priority, milestones, and even tags. The task dashboard allows you to see what is current / past due at a glance, for all workspaces, all tasks, all users. There are also several sort options.

Weblinks
I don’t use this very often, but it is basically just a repository for links related to the workspace / project.

Time
Time reporting within openGoo needs some work. It is actually a bit buggy, but the way that time is assigned to tasks is actually very smart. I am looking forward to what they do to resolve the issues with the reporting functionality. For now, I export all of our time out of mySql using a small php report I wrote.

One of the things that I really enjoy about openGoo is that they have a very fast development cycle. I started messing with it at version 1.0 about 3 months ago, and they are already on 1.4.0 rc2. I’m amazed with how quickly they have been moving through releases. The community is a little weak / inexperienced, but the staff who are working on the project are very good about answering questions in the forums. This is huge for an open source project that is used for commercial purposes, especially for something as important as project management.

I’d like to see how they handle the billing capability within it though… right now I’m not so happy with the fact that it does not allow you to set per workspace billing rates or per service rates either. All billing rates are based on the user performing the work. This also does not take into account your actual cost as a development firm. The way that they handle accounting within openGoo is not the greatest, but I think that they will improve upon it.

All in all, openGoo is one of the best collaborative platforms I’ve used in a long time.

About Us

"Managed Mayhem" is a software project development and management blog maintained by Jim Rising. Jim Rising is an Adobe Cold Fusion developer who lives in Murfreesboro, Tennessee with his wife Melissa, their son 'Haven', cat ‘Rusty’, and dog ‘Güenther’. He currently freelances from home.