Agile Offshoring : It's hard work but it works!

Discussions

News: Agile Offshoring : It's hard work but it works!

  1. Agile Offshoring : It's hard work but it works! Software Off-shoring is a reality of the day however there are many projects which fail due to incorrect off-shoring. Apart from tremendous advantages, off-shoring brings additional complexity, risk and avenues for wastage. This experience report will discuss how we turned off-shoring into a successful model based on Toyota manufacturing Process. We call this methodology 'Lean Agile Off- shoring.' Our goal to go offshore was to deliver software faster by leveraging a vast talent pool at offshore location with higher efficiency. We chose Scrum and the Toyota way of working because of its strong foundation in successful new product development. The Toyota principles can be very well applied to any software development project. The underlying points discuss the Toyota way of working and how they were mapped to make our off-shoring process lean. The overall methodology remained Scrum and Toyota way of working helped us to look at improving on a daily basis. The project on which we applied this methodology is a batch processing system for a central social security organization. Our system processes relevant incoming data employer and employee data from the Income Tax department, validates this data and creates income declarations for a person for a period. The system handles 16 million records per month with the throughput of 50 records per second. Principle 1: Define a long term philosophy and stick to it We started out with the project off-shoring with one thought in mind that we are going to work on this not like a one project task but we are going to base all our actions on a long term vision to be the best agile company with a stable and repeatable off-shoring model. It should be a model from which the software community can benefit as a whole. We would not take any shortcuts that defy or compromise quality for the project. We would actively invent and refine the way to work so as to improve the success rate of the way agile projects are off-shored across the industry. Principle 2: Create connected, continuous process flow to bring problems to the surface We decided to be agile in all our actions and follow no plan driven methodologies. However it is easy to lose way without a plan, this is where continuous communication and integration play a vital role. We followed a model of regular sending of ambassadors from one site to the other, making the business context travel across geographies, building trust, gelling across the wire and mentoring. Communication was also facilitated by having 'always on' communication machines across geographies. Talking to the team across the geography was a matter of picking up the mike and talking. For project work we had single code base for multi site development with continuous integration so that problems would surface out quickly and would be taken care of immediately. Principle 3: Using pull systems to avoid overproduction Taking hint from the above principle we decided not to produce everything based on push but on the basis of stakeholders pull. After all we are not writing lines of code to increase our code base but the whole intent is to add business value with every line that we write. We reduced the size of the iteration if the pull was not strong enough or else used time to refactor, improve the quality of process and deliverables thus achieving success in making sure that we produced what the customers want, when they want it, with the best quality. Principle 4: Leveling burden (heijunka) Toyota talks about reducing muda (waste), muri (overburdening people) and mura (unevenness). Keeping in mind the above 3 the biggest wastages in an off-shored project are in terms of extra features, more detailed than required requirements, extra layers of abstraction between the team and the customer, finding relevant information, defects not caught by tests, inefficient hand-off to the customer. We took care of eliminating waste by developing only for todays stories, using story cards which were detailed only for the current iteration and had just enough information, coded directly from the stories and for clarification even the offshore team could always be in touch with the customer, test first both developer and customer tests. For leveling of work we always worked according to the team velocity and made sure that there were no uneven trends in the amount of user stories completed. There is nothing more harmful in a software project than burnt out team members. Teams velocity plays a very important role in deciding who does what and how much we do as a team. Principle 5: Build a culture that stops to fix problems (jidoka) We promoted a culture to stop work if there is a problem that can affect the quality of the deliverable. If the offshore-onsite team felt that they could not communicate effectively we installed an 'always on' communications machine with always-on skype voice and video channel. Once the team felt that our sprint evaluation was not being effective and we were not getting what we desired, we called off the evaluation, brainstormed on how we can do it better, had an improved kickoff in which we reduced the time spent on the reviews and followed an agenda to guide us through. If there was an issue with our continuous integration or performance testing environment then we fixed that first before going further with developing stories. If we developed enough stories but the functionality testers did not have time to test them then we stopped developing more stories till the testing team was comfortable with what we had done so far. All this not only helped us by using counter measures but we were also able to error proof future situations. Principle 6: standardized processes and procedures Standardization is the basis for continuous improvement, empowered employees, rules and procedures as enabling tools, and a hierarchy that supports organizational learning. We created standardized processes for the way to do development using TDD, issue tracking and resolution process, build generation and testing process. This is not to say that these processes were frozen in time, they were as alive and agile as our project but having some standards ensured that we started on a stable platform which can be improved further by the team. Principle 7: Use visual control so that none of the problems are hidden Our motto, everything should be visible to everyone on the team and yes this includes the client. We created a common product backlog for onsite and offshore team in Jira, created a common burn down chart and issue log which was available for the client to report production issues and even look into our daily status. Cruise control would visually report the status of a build with every check-in and a build bunny would announce the success or failure of a build. The physical and virtual walls and white boards had enough visual information for any stakeholder to walk to the wall and white board and get the information about the project in a matter of 10 minutes. We decided to keep all visual information on our virtual team boards which were created on a wiki and then print out the relevant ones to paste on team walls. Principle 8: tailor technology to fit people and processes The truly lean project has two key features. It transfers the maximum number of tasks and responsibilities to developers actually adding value to the product, and has a process for tracing defects and working on them as soon as they occur. People is the greatest asset of an agile team, we want the team to change and tailor technology to suit their needs best rather than technology dictating how things should be done. In one example we moved our entire presentation logic from Struts to Spring MVC because it made more sense in the present context and the team as a whole took a decision to make the change. The best decisions and commitments came from a team when they took the decision on their own. The self organizing team knew best how to tailor technology and processes to work in the best interest of everyone. Principle 9: Develop leaders from within the team It is a well-known concept machinery depreciates and people appreciate. We were committed to develop leaders in our offshore project from within the team. Instead of bringing scrum masters from elsewhere we sent some people from onsite and offshore teams for scrum master training and needless to say that these are the people who know the teams best. Principle 10: Develop exceptional team associates Unhindered communication, effective teamwork, form vs function of teams, good pay, the best facilities, a safe working environment, a work balanced life, continuous improvement, job rotation. All these when followed in the right spirit can create star teams. Healthy communication within different geographies is a core for efficient teams. For this we had a lot of seeding visits early the in the project intended to create the relationships, and regular visits to maintain the relationships. Of course the idea of these visits was not to get work done but to get healthy relationships going between geographies. People are often discouraged from asking questions, talking about problems, warning about unfeasible deadlines, or proposing alternatives to perceived instructions from superiors. Though getting teams to be more pro-active is an uphill battle, and one that inevitably takes a lot of time. However we actively promoted a culture of asking a lot of questions and highlighting issues. Once people realize they have the freedom, and the responsibility, of making decisions - they further contribute to becoming an exceptional associate. Principle 11: Develop partners and suppliers as extension of the enterprise Fair and honorable business relations is the key to have long term working relationships with the partners and suppliers. We tried to follow this in spirit by exposing the wiki and Jira to customer and other software vendors who were working on different modules of the same project. This have full visibility into who is doing what, we could set clear expectations with all the stakeholders and vendors. The advantage, we built a huge wealth of trust and transparency, we had nothing to hide. This not only gave a lot of credibility to our offshore process as a whole but actively helped in our long term mission to take the best practices to the software community by involving a bigger group of partners. Our partners learned from our burn down charts and Sprint signatures and we learned from theirs. Principle 12: Go and see for yourself There is nothing like being in the trenches to get the real feel of the situation, hence we had no lip service designers and architects on the team, everyone had to code and code well apart from any other role that they were playing. Period. This helped us bringing about the attributes of clearly assign tasks to yourself and other people, speak on basis of well verified data, consult experts, analyze and understand situations and their solutions. Principle 13: Evaluate alternatives while looking for consensus and implement them fast We actively followed a policy of not picking a single direction and going down that one path until we had thoroughly considered alternatives. Once we picked the right path, we moved quickly and continuously down the path. At several instances we tried to find alternatives to various issues like how to get Dutch documents across to the offshore team - should we manually translate them? Should we use a tool, since a tool would get us 60% there? Finally we decided that the Dutch team would make Jira issues out of relevant documents and then the offshore team would study the issues and ask questions on that. This approach went well and quickly with the team. Principle 14: Become a learning organization through relentless reflection ( Hansei ) and continuous improvement (kaizen) This is the most important principle which has helped us reach where we are today, we do rigorous iteration evaluations for reflections of good things to preserve and areas of improvement to help us improve iteration on iteration. We solicit constant and immediate client and team feedback so that the ultimate aim of the software is realized in an effective and efficient way. Apart from this we made it a point to get to the root cause of any problem that we came across by asking “why” 5 times. Conclusion To conclude I would say that we are still learning. The lean production metaphor has been successfully applied to software offshoring but it can be improved. The Toyota principles when applied to software offshoring can make remarkable differences to the way software is being developed and offshored. We are trying to continuously improve and reach our goal of 'Whitebox Lean Agile Offshoring' in which transparency and quality are taken to the extreme. As an advice for the offshoring industry, follow Scrum with the Toyota principles in spirit without diluting their essence. Apply them to your way of working and see the magic unfold.

    Threaded Messages (27)

  2. double negative[ Go to top ]

    This concept doesn't really hold up in practice from the few companies I've met that do it. There software is always an abysmal failure of engineering. Like everything else in the commercial world - the same holds true here. You get what you pay for.
  3. Find agile offshore team[ Go to top ]

    I think agile offshore is the right answer to small/medium sized companies. For big players like IBM, they are easy to setup Dev Center in India and China, and hire several thousands talents for distributed development. But for small software dev organization, how to get help from distributed dev team in the flat world (also means from low cost countries ). I think most of the small team are following agile way, so how to find agile offshore team is the fist step.
  4. Too generic statement[ Go to top ]

    There is always abysmal failure in every engineering aspect . Leaning towards agile methodologies mitigate the risks of sudden surprises .
  5. Any person with business education will tell you right away that the described system has very little to do with Lean/Toyota Production System and is a geek-read-something-on-Wikipedia version of it. Lean is all about the elimination of the waste (non-value-added activities). Having the overhead of development team sitting so far from both technical leadership and the client is a huge waste right there that can not be eliminated by definition. Another big deal in Lean is valuing people. Offshore teams are almost always built on temporary bases, have high turnover and hence are fundamentally ill-suited. Having team so remotely is a big problem for Agile, as well, which is all about close communication. Yet, whatever they used - maybe it worked, even if it is neither Agile nor Lean. It would be interesting to hear the feedback from the actual customer of this group. cheers
  6. That's great! I think you are tremendously fortunate because offshoring is incredibly challenging. I think this is definitely the way to go, however, there are many issues that offshoring introduces as well, and many of the problems arise due to the cultures that already exist. One mistake I see many companies making is the commoditization of developers. Of course I am biased, but all things considered, you go to one extreme which I think for pure delivery, is the ideal: everything in house, developers, QA, Product, everyone working in a collaborative team environment with hyper productivity. There is also another extreme where you move delivery responsibilities completely offshore. Often times, I think this happens because development is too much compared to hard manufacturing analogies and the pure soft art (communication, thinking, abstraction, etc) of development and delivery are undervalued until it is too late. Then the fall out comes in terms of quality, flexibility, time to market, etc. There is definitely a balance, and it sounds like you have achieved something wonderful in how you are managing the delivery of the teams. This is very commendable, and incredibly hard to do, especially having a culture that supports transitory staff turnover, changing requirements, business shifts, ... One analogy I use often in response to looking at software development as a pure body count is sports: in the NFL you hit an age and then peak, and if you are lucky, you stay in as a coach. In software, the older you get, your experience widens, and you become much more valuable to the business, not just in architecture, development, strategy, but also in the business domain, relationships, and many other factors that are very tangible. SCRUM and TPS/LEAN are great. I wish that many more teams were able to get the executive support to successfully integrate these whether offshore or on. I also wish that more companies would take a Global View perspective combined with a localization/point of presence strategy as opposed to pure accounting and resource shifting. McDonalds (say what you want about the food ;-), is a great franchise, the happy meal toys are rolled out simultaneously across the world. They localize the service, and collaborate activities, but they are not headquartered offshore, and I fear that too many companies are going the oppositie direction by thinking that they can completely offshore services and development for cheaper labor, often times into immature local industries (relatively speaking). If you believe some of the figures that are coming out, more and more projects are being brought back in house in large global companies. Congratulations, what you have achieved is tremendous.
  7. Certain aspects mentioned in the article I would like to get some clarification. "by leveraging a vast talent pool at offshore location with higher efficiency" -- where are these resource ...i mean when all the companies are having such a tough time to get good resources where are these people and if they exist then why the companies are having a tough time finding them? "Apart from tremendous advantages" -- what are the advantages? This article shows clearly that offshoring adds tremendous amount of overhead to any project.
  8. "i mean when all the companies are having such a tough time to get good resources where are these people and if they exist then why the companies are having a tough time finding them?" Well here is the deal: if you are a blue chip compan opening an office in Inda, talent is easy to find. Indians are very brand conscious when it comes to choosing employers. If you are not a bluc chip company, you are better off partniring / hiring a blue-chip outsourcing company. Take your pick -- IBM, Accenture, Bearing Point, CGI, Infosys, TCS Cognizant etc. etc. Advantages #1. Cost #2. Experience : if you are building yet another enterprise application, why not go with someone who has done this several times. This is the same consideration when you outsource development to a consulting company. #3. Added Scope : Sonce you have reduced your expenses, you may want to do more than bare minimum that you budhet allowed you earlier. ( yeah --- this is a rehash of #1) #4 Risk Management : The project risk is owned by vendor if you are smart about the contract. There may be more -- but I gotta go for now.
  9. Right exactly bad mASH. These guyz r obviously expertz in the outsorcin area. They know stuff. What more do you want ? Also please check out my own company - we do a lot of the same things. http://www.huhcorp.com/
    ...#4 Risk Management : The project risk is owned by vendor if you are smart about the contract.

    There may be more -- but I gotta go for now.
  10. This article shows clearly that offshoring adds tremendous amount of overhead to any project.
    There is nothing denying that it can add overhead to the project but the secret lies in doing it in the right way to make the advantages outweigh the pain. What we tried to do was probably "not different things but doings things differently" a.k.a the right way. In terms of advantages if you can maintain the quality then offshoring has tremendous advantages apart from the ones mentioned by "bas mASH" in terms of 1. Cost, just one part of the equation 2. Increase in productivity. 3. You can scale up your size easily with locations like India, Russia etc. 4. The time to market is reduced with virtually 24-7 availability. 5. The company can expand it's reach. 6. Can come closer to global customers. For the well documented success stories refer to Jeff Sutherland's case study http://www.infoq.com/news/SirsiDynix-Case-Study and http://www.martinfowler.com/articles/agileOffshore.html#CostsAndBenefitsOfOffshoreDevelopment
  11. Please check the link[ Go to top ]

    I was going through the article in the link http://www.infoq.com/news/SirsiDynix-Case-Study. The 56 member distributed/outsourced team was split between Provo, Utah; Denver, Colorado; Waterloo, Canada; and St. Petersburg, Russia. Doesn't look like a outsourced to cheaper location kind of stuff. It is more like experts around the world collaborating and sharing ideas to reach a solution rather than giving work to somebody who has no idea of your business and expecting that the result will be delivered in time and bug free.
  12. People are still offshoring?[ Go to top ]

    The biggest news in this article is the suggestion that offshoring is still perceived as successful by some people. I'm hard pressed to find a single example I'm aware of that was remotely successful. Usually it's a great way to build software of lower quality, in longer time frames while spending all of your 'savings' on additional middle management to deal with the colossal headache that is off-shoring. But you can still find folks who think EJBs are the greatest thing since slice bread so I guess word travels slowly.
  13. I'm hard pressed to find a single example I'm aware of that was remotely successful. Usually it's a great way to build software of lower quality, in longer time frames while spending all of your 'savings' on additional middle management to deal with the colossal headache that is off-shoring.
    Choosing a cheapest fixed price solution with a blurry specification always lead to the situation you describe. But if you thoroughly described the requirements (or agreed on a TM scheme), hired a technical offshore coordinator like me :) your are almost safe.
  14. After almost a decade in the industry and around 15 projects of varying sizes, I am yet to see a single project that didn't have a "blurry specification" at best. Even if people thought at the outset that the specification was thorough and precise, it ended up being blurry. The problem with software, being a semi-abstract product is that people never know what they want before they see it and can use it. / Wille Blog: Buzzword Bingo
  15. I've read an interesting entry on your blog:
    The moral: well, if you are going to offshore or outsource something, you better offshore the whole picture, not part of it, because otherwise it will fail hideously. There are no savings to be made by deluding yourself that you can replace one expensive guy doing a job with ten cheaper ones. The communication overheads just rise exponentially. If you are going to replace something that already works for something cheaper, you better replace like for like with the full picture in mind, and that is rarely the case with the modern drive to offshore.
    Completely agreed. In my situation, customers wasted huge money for implementing a bloated poorly documented framework in their country by a bunch of trusted consulters and then try to make others develop anything using this peace of ... work. The funniest thing is that developing an abstract framework is about 75% of the project budget. This is especially popular in the Java world. If you want to outsource the project, do it completely and hire a company you can trust ;) although it's apparently hard to achieve. Regards, Fyodor Kupolov Make ETL Easier - Download Scriptella ETL.
  16. myth of vast offshore talent[ Go to top ]

    I think those in favor for off shoring over sell the idea of "vast offshore talent". Like anywhere else, the percent of top notch programmers is the same. It doesn't matter which country it is. There may be a greater number of top notch programmers round the world in total, but the percent remains relatively stable. From first hand experience working with a large offshore team, finding talent is hard. Once a talented engineer gets experience, they move on, so the problem of retaining talent is still present. But we all know why offshoring is increasing. It's not about finding talent, it's about cheaper costs. The savings then go to the executive's salary. I would think that having an environment that encourages good communication and planning is obvious and doesn't need to be rehashed. my bias 2 cents. peter
  17. I think those in favor for off shoring over sell the idea of "vast offshore talent". Like anywhere else, the percent of top notch programmers is the same.
    Outdated computer science courses, academic institutions infested with infights among professors, political interference and no accountability has left very few educational institutions in India that produce 'programming talent'. Students get very little support from the academic institutions and industry to learn and improve. In India summer jobs for IT students are either not available in the IT industry or filled up by the people having a relative in some influential role in a company. The result these programmers have no clue about the programming practices. Their graduation projects are minor variations around similar topics like chat application, library management system and student registration system over the years. I think that the talent can be grown in the academic institutions and industry. Programmers can be tought to produce quality software for business software development. I see candidates at borderline during recruitment, who can be trained to become more useful than presently. I bet that the world would have been worse off than now if every graduate of a university system would have the talent of Einstein. India has vast untapped talent but unless it is cultivated 'vast offshore talent' remains a myth.
  18. It looks like you were working in a perfect world, where everyone one was coding and that too good quality coding, client was always ready and had time to work with the offshore development team (client must have been also understanding and following agile practices), people had work life balance(don't know if this statement of yours is really truthful). I do agree things do work but I feel they should work with other techniques of development as well if the world is as perfect as yours.
  19. This is a very well written article . Principle 8: tailor technology to fit people and processes This is the key most of time . The technologies are rarely a reason for failure on large or samll projects .
  20. We chose Scrum and the Toyota way of working because of its strong foundation in successful new product development.
    This is essentially what the Poppendieck's describe in Lean Software Development. It would benefit any team regardless of whether they were co-located or spread across the globe. If you try to use offshore people with a more traditional "ship the requirements and wait 8 months for a system" development model, you are going to fail albeit for possibly less money than with local people. Dave Rooney Mayford Technologies
  21. I get scared thinking of a car assembly line where we have to wait for tyres to come from Country X, Stearing wheel to come from Country Y and technology to come from Z. Just because Toyota is # 1 doesn't mean that we can apply Toyota techniques to software development.
  22. I get scared thinking of a car assembly line where we have to wait for tyres to come from Country X, Stearing wheel to come from Country Y and technology to come from Z.

    Just because Toyota is # 1 doesn't mean that we can apply Toyota techniques to software development.
    You're taking the automotive manfacturing metaphor too literally. I encourage you have a look at Mary and Tom Poppendieck's work on Lean Software Development. It bridges the gap between the Toyota Production System and the IT world. Their web site is http://www.poppendieck.com/. Dave Rooney Mayford Technologies
  23. I get scared thinking of a car assembly line where we have to wait for tyres to come from Country X, Stearing wheel to come from Country Y and technology to come from Z.

    Just because Toyota is # 1 doesn't mean that we can apply Toyota techniques to software development.
    I doubt if you have gone though the whole article of Vikas. It seems that you have poor understanding of manufacturing as well as the decades old problems of software development. In an Assembly line 'WAIT' is a bad word. People lose jobs when they wait. Toyota model is about reducing/eliminating the waste caused by 'WAIT' and rework.
  24. It seems that you have poor understanding of manufacturing as well as the decades old problems of software development.

    In an Assembly line 'WAIT' is a bad word. People lose jobs when they wait. Toyota model is about reducing/eliminating the waste caused by 'WAIT' and rework.
    With all due respect, one should not criticize somebody when he is not sure what he is saying, himself. If you were more familiar with the Theory of Constraints, you would know that "wait" is only "bad" on the constraint (bottleneck). On non-bottlenecks wait is expected and normal. Actually, according to Eliyahu M. Goldratt, the father of TOC, a plant where everybody is working all the time is inefficient. Such plant will create hugely excessive amounts of inventory and eventually, inevitably sink into bankruptcy.
  25. WAIT is not bad[ Go to top ]

    WAIT is in fact not so bad. Industry support is lacking for developers and that is because industry is not interested in training people. They just want to get ready made solutions. For people to keep pace with technology investment has to be made in training programs and developers should be allowed creative time. Whenever I met people working in outsourced project people complain about stagnation. By adding the overhead of outsourcing on the developers so much time is wasted in trying to communicate and coordinate that very less time is left to refresh your skills. There is no time left to learn new skills. Toyota model is about encouraging people to come up with new ideas and to implement them. And that's why they are producing cars that keeps our total cost of ownership less.
  26. I disagree with
    Just because Toyota is # 1 doesn't mean that we can apply Toyota techniques to software development.
    and I agree with Dave when he says
    It would benefit any team regardless of whether they were co-located or spread across the globe. If you try to use offshore people with a more traditional "ship the requirements and wait 8 months for a system" development model, you are going to fail albeit for possibly less money than with local people.
    I think the main idea is to give it a try first! Without trying you should not sweep it under the carpet. Once you try then you would see the results for yourself. Franklin D. Roosevelt once said, "The only limit to our realization of tomorrow will be our doubts of today."
  27. Offshoring is tough if you are doing it for reasons other than just cost-saving by using commoditized resources. It is tougher if you are really keen to reap the benfits of agility. This primarily happens because without co-located teams, one of the basic principal of scrum, "Frequent inspection and adaptation" is far more difficult to achieve in practice. It would be interesting to know how exactly you were able to achieve this. Some actual examples or scenarios would be helpful. Co-location does not mean that all the members of the both the teams have to be co-located. One common solution suggested is to make more frequent visits from both sides. Though this helps, it is not a complete solution. There is a need to grade various roles in terms of extent of closeness required. Your experiences on this aspect would provide good pointers for others.
  28. This primarily happens because without co-located teams, one of the basic principal of scrum, "Frequent inspection and adaptation" is far more difficult to achieve in practice. It would be interesting to know how exactly you were able to achieve this. Some actual examples or scenarios would be helpful.
    I totally agree with you that it is hard when the teams are not co-located. That is one of the reasons for naming the article the way it is now. I am working on sharing the different communication techniques that we followed and how we achieved the best possible virtual co-location. Keep watching this space :)