It might seem obvious to some, but it’s a topic worth addressing clearly: “Agile development is not well-suited for outsourced development projects undertaken by Japanese SIers (System Integrators, システムインテグレーター).” In this post, I will outline the reasons for this conclusion.
To provide some context, I work in a department of an SIer that primarily handles AI-related projects. My responsibilities include data analysis, Proof of Concept (PoC) for model creation, and system development for AI model operations. In AI projects, it’s often impractical to adhere to a traditional waterfall approach, so we end up adopting a pseudo-Agile development method.
1. Why Waterfall Doesn’t Work for AI Projects
The primary reason waterfall development doesn’t work in AI projects is that defining the requirements for an AI system during the contract phase is virtually impossible. Requirement definition in AI projects demands significant knowledge of AI, data science, and the business domain. Moreover, the performance and characteristics of AI systems remain unknown until PoC testing is conducted. Neither the client nor the vendor possesses all the necessary information at the outset.
Consequently, rather than following the rigid sequence of requirement definition → design → implementation → testing, the project progresses iteratively. This involves running multiple smaller projects to gather information, evaluate whether to continue or withdraw, analyze the business processes AI will support, prepare the data, develop the AI model, and build the operational system—all in parallel. Without this approach, the project would be too uncertain to even establish a contract.
As a result, the project naturally adopts a Scrum-like, Agile framework, dividing work into sprints, managing requirements through a backlog, and advancing incrementally.
2. Why Agile Doesn’t Fit Outsourced Development by SIers
If the project is handled internally within a company, this Agile-like approach might work. However, as a vendor executing the project under a fixed-price contract (請負契約) and the client’s direction, it’s unrealistic to implement Agile effectively. To be clear, this discussion focuses on outsourced development, where the client provides the requirements, not internal development of a company’s own products or systems, which often suits Agile better. Below are the key challenges of applying Agile in the context of Japanese SIers’ outsourced projects.
Communication Overhead (Especially Meetings)
The most critical issue when implementing Agile in SIer-led projects is the overwhelming communication cost. In many cases, communication itself becomes dysfunctional. Agile development demands real-time, frequent, and synchronized communication. The more Scrum-like the approach, the more meetings are required.
For instance, Scrum necessitates frequent meetings between the Scrum Master (SM) and the Product Owner (PO). However, if the PO role is assigned to a vendor’s project manager or salesperson, the process often fails. Without decision-making authority, the vendor’s PO must consult with the client after meetings, leading to delays and overturned decisions. This halts development and stalls progress.
The PO must be a key decision-maker from the client company, not just a representative. Additionally, meetings like daily scrums, sprint planning, and sprint reviews require the participation of all stakeholders, significantly increasing the cost of meetings (meeting time × number of participants).
In one of my experiences, two-week sprints involved:
- A weekly reporting meeting with all stakeholders (combining sprint planning and review).
- Two weekly meetings with the client’s SM and the vendor’s PM and PL.
- Two weekly meetings among the vendor’s PM, PL, and team members.
Preparing for these meetings was equally time-consuming. For example, I spent over a day preparing materials for the weekly reporting meeting, attended the half-day meeting, and then spent another half-day revising documents and creating minutes. The remaining time was devoted to scripting, model training, and preparing results for the next meeting—all while juggling other projects. This led to a “death march” (デスマーチ) situation that lasted nearly two years. The project eventually ended prematurely due to burnout among key stakeholders, with team members leaving or distancing themselves from the project. Despite having the budget, the lack of willing participants rendered the project unsustainable.
Misaligned Team Objectives
Aligning the purpose and mindset of a diverse team is another challenge. Agile, particularly Scrum, assumes a high level of collective commitment and enthusiasm, which often doesn’t align with reality in outsourced projects.
Clients may view the project as just one of many they are managing, lacking personal investment. For example:
- A project initiated by a previous manager that the current one doesn’t deem necessary.
- A system ordered to improve operational efficiency but overshadowed by the client’s day-to-day workload.
Similarly, developers are often motivated by factors outside the project, such as personal interests or external responsibilities, rather than the project’s success. In SIer projects involving multiple companies, including subcontractors, aligning objectives across organizations is virtually impossible.
Dependency on Individuals
Agile often prioritizes flexibility over documentation. However, in inter-company projects, legal and contractual requirements necessitate substantial documentation. Even so, rapid changes in requirements often leave documentation lagging behind.
This creates situations where critical knowledge resides solely in individual team members. For instance, data analysis results, data preparation guidelines, and model development processes may only exist in the minds of specific contributors. If these individuals become unavailable, the project risks stalling or collapsing.
In SIer projects, personnel assignments are fluid, with team members frequently reassigned to other projects. Subcontractors and temporary engineers also rotate regularly. This creates a high risk of losing critical knowledge when individuals leave, leading to wasted efforts and project resets.
Ambiguity in Deliverables
Agile projects, particularly in AI, often struggle with defining clear deliverables. At the PoC stage, the feasibility of deploying an AI model or even the specifics of its operational system are usually uncertain. This makes it challenging to determine what the project is ultimately expected to deliver.
One workaround is to define reports or progress on sprints as deliverables under a time-and-materials (T&M) contract (準委任契約). However, without clear objectives, projects risk aimless progress, consuming time and resources without tangible outcomes.
For both vendors and clients, this creates a burden of continuous decision-making to refine project goals, which can lead to frustration and fatigue.
3. The Need for a New Approach
In AI projects, both waterfall and Agile methodologies have significant limitations. While hybrid approaches combining elements of both are often suggested, I believe more fundamental changes are needed. Establishing a method distinct from both waterfall and Agile development may be necessary to address the unique challenges of AI projects and outsourced development by SIers.
Although what this new method might look like remains unclear, continued exploration and learning are essential. If I discover promising ideas, I will share them here in the future.
Thank you for reading.
Comments