Selecting Your First Big Data Project – Planning for Success


Note: We are currently posting the top 5 posts of 2012, this week through Jan. 1. This post is #2 on the list and was originally published Apr. 10, 2012.

So far in this series, we looked at how to determine the compelling drivers for your first big data project, the importance of choosing the right people for your team, and in the last post we examined the computing environment. This final installment talks about planning for success.

Boil a Bathtub

There’s an old saying, “You should boil a bathtub before you try to boil an ocean.” Humor aside, this is actually a very important point, and one that people sometimes lose sight of. Because big data technologies offer profoundly new ways of doing things, we oftentimes see customers that are starry-eyed on very big ideas.

Big, transformative ideas are important to your business, and they should play a key role in your strategic plan for what you want to accomplish over the next 5 to 10 years. But those ambitious ideas are not the right place to start. Your initial objective should be manageable and should have a clear, direct line-of-sight ROI. I covered this point in part 1 of this series.

Some of this can be accomplished just by practicing good project management discipline, but it’s important to note that any journey starts with the initial steps of being specific about the objectives upfront.

Good project management aside, establishing an initial set of attainable objectives is especially important with big data technologies because of the organizational dynamics we see in people who have inflated expectations. The technology can open many doors and address previously unsolvable challenges, but it cannot overcome hyper-inflated expectations.

Before you start, get a clear, agreed-to objective that is fairly modest so the team can gain experience while having a strong likelihood of success.

Plan for Success

This next situation is what we would call a “high-quality problem,” but it is still potentially a problem. It is common that once the project is rolling and you have initial results, the project sponsors are going to ask you to put it into production faster than you might expect – or want. To ensure that this does not catch you by surprise, be prepared with a plan that describes how you would scale the project in a way that does not blow your ROI case.

The line of business constituents you are supporting will become very impatient. They will want to use – and expand – the project as quickly as possible so they can take advantage of the insight that these projects often deliver. Plan for it. This is one of the reasons why you want to pull/touch/push from your other key systems during the pilot.

Keep your original set of agreed-to objectives close at hand and remind people of them as needed. Remember, you have to be able to boil a bathtub before you can boil an ocean.

Once again, here are links to the other posts in this series:

What do you think? Are you ready to choose your first project? If not, what else would you like to know? Leave me a comment and I’ll try to help.

Smarter Computing Analyst Paper - HurwitzTo effectively compete in today’s changing world, it is essential that companies leverage innovative technology to differentiate from competitors. Learn how you can do that and more in the Smarter Computing Analyst Paper from Hurwitz and Associates.

Subscribe to the Smarter Computing Blog

Recent Posts

Reach the forefront of digital transformation with waitless computing

Andy Dominguez

That’s why IBM Power Systems continue to introduce products and services that make waitless computing a reality. On October 5th, tune in for the new advancements Power Systems is bringing to the market—and an entirely new way to consume Power Systems.

Continue reading

VersaStack expands to more applications and workloads

Eric Herzog

Today, IT departments are caught between two conflicting forces: huge demand for new applications and flat IT budgets. VersaStack can efficiently reconcile the conflicting forces of application and workload demand and cost, while enabling more business value and expansion. IT departments will transform from cost burdens into enablers of innovation and business growth.

Continue reading

5 Responses to Selecting Your First Big Data Project – Planning for Success

  1. Kulbhushan Sharma says:

    Hello Tom,

    I read the entire series just now.

    I represent an organisation which still makes a living from sites like elance and odesk. We have solid experience in the conventional ways (SQL, CRUD) and we understand the need for Big Data.

    In part-1 you talk of ROI and not to take up a academic project. But for us we will have to do it that way — as in this market it is a Chicken-Egg situation you do not know a technology you do not get a project.

    How can we get out of this — do you have any suggestions of a Big – Data project (may be/ should we analyse logs of our servers running conventional software) or do you think this would be farcical.

    Thanks & Regards,

  2. Tom Deutsch Tom Deutsch says:

    Hi KBS – thanks for the comment/question. Sorry I am running a bit behind here, lots of customer meetings.

    No doubt that there are chicken/egg situations in the real world, but it sounds like:

    1) You need to determine if there is market demand, after all while learning for learning sake is great I assume this is a for-profit thing

    2) If there is demand (which there is) then what is the profile of skills/capabilities that are a good fit for what you already do. Look for an adjacent space to leverage skills, contact and domain expertise

    3) If you pass #1 and #2 (which can be fast to review/ decide upon) then you get to a point where you need to learn the skills to engage/fulfill the market demand.

    There are several training and skill development options out there. Starting with to develop some background is a great way to go.

    I'd also encourage you to play in Cloud environments. If you have a relationship with an existing customer that is open to this sort of thing maybe you could jointly pick a project that matters to them (back to the ROI!) and do the work at a reduced rate to get started. You win experience, they win the ROI, and off you go.

    Did I fairly answer your question?

  3. Kulbhushan Sharma says:

    Thanks Tom, your answers will help me keep looking at our current scenario to bring in "big-data" needs where we think are appropriate especially "jointly pick a project that matters to them".

    Thanks & Regards,

  4. I think experience is what really counts; people want to know if you are actually good at managing projects. These days, there is an urgent need to manage projects in all sorts of industries.

  5. Tom Deutsch Tom Deutsch says:

    Hi Edward – thanks for the note. Agree that skilled and savvy project managers are more needed than ever. Thanks for touching base.

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>