7.2. The Mainline Model

A codeline is, for the most part, the same as a branch. But though the term branch can mean any set of files created by branching, codeline is imbued with slightly more significance. Codelines have a purpose, a strategic role in the development of software. Together, codelines form a model of software evolution.

In the Perforce view of software configuration management, one model—the mainline model—is most effective. This chapter discusses codelines and software evolution in the context of the mainline model. It's not a Perforce-specific discussion, by the way—the mainline model is a general concept, not a Perforce feature. But it is the concept on which much of the design of Perforce is based.

7.2.1. From ideal world to real world

In the ideal world, there are no bugs, no schedule crunches, no personnel changes, no market shifts, and no technology revolutions. Software in the ideal world is simply developed and released—that is, new features are developed, and when they're ready, a new version of the software is released (see Figure 7-2.). Each release contains features that work perfectly.

Figure 7-2. The mainline in the ideal world

If there were such an ideal world, we probably wouldn't need an SCM system. Even so, we'd have a collection of files evolving together in a codeline. This codeline embodies the evolution of our software; it is our mainline. In the ideal world it would be the only codeline we'd ever need.

A sad fact of the real world is that the software we develop isn't perfect. Because of that, we subject software to a testing phase before release, during which bugs are invariably found. We could do all this in the mainline if development could halt for the testing phase, and if all bugs could be found and fixed during testing. But all bugs are not found during testing; many are found after software is released. And we can't hold off on new development during the testing phase because we have deadlines and market pressures to face. So we branch completed software from the main codeline into a release codeline.

Branching release codelines allows us to do two different kinds of software development at once. One kind is bug fixing—euphemistically known as stabilization —and the other is new feature development. In the release codeline, we stabilize a version of our software—both before and after release—while in the mainline we get on with developing new features. As we stabilize the release version, we can make point releases—that is, we can rerelease the software—without the risk of releasing untested new development.

Another problem with the real world is that our customers expect us to fix bugs in old versions of our software even as we are developing and stabilizing new versions. To deal with this, we branch a new codeline for each release, leaving our old release codelines intact for more bug-fixing. Now the typical shrinkwrapped software evolution model begins to take shape. A mainline charts the course of the overall development, while release codelines sprout as new versions are ready (see Figure 7-3). When releases are no longer supported, the codelines designated for them cease to evolve, but the mainline persists.

Figure 7-3. Release codelines

In the ideal world, all development projects are completed on schedule. No matter how many new features are slated for the next release, developers in the ideal world get them all done on time. In the real world, a single incomplete project can hold up an entire release if it's in the same codeline as completed projects. To decouple development projects from one another in the real world, we can branch the mainline into one or more development codelines. Development projects are delivered to the mainline as they're completed, and when enough development is completed to warrant it, a new version is branched for release stabilization. (see Figure 7-4).

Figure 7-4. Development codelines

Thus our mainline evolves as new development is completed, although development does not necessarily take place in the mainline.

And there you have it. The mainline model is a necessary deviation from the ideal world. It seeks to preserve the intent of the ideal world while accommodating the constraints of the real world.

Why We Don't Drive Through Hedges Why not just branch a development codeline into a release codeline? Or merge a bug fix straight from a release codeline into a development codeline? Well, just as there is no law of physics that keeps you from driving through hedges to get on and off the freeway, there's nothing in Perforce that keeps you from integrating changes any which way you please. One has only to look at traffic on a freeway to see why entrances and exits are controlled. Clearly, driving would be inefficient and unpredictable if they weren't. Because we can't see the flow of change, it's not so easy for us to understand that we must control change for the same reason: parallel development would be inefficient and unpredictable if we didn't. Think of the mainline model as the freeway system of parallel development. It's a fast and reliable way to get somewhere, but only if we resist the temptation to drive through the hedges.

7.2.2. The flow of change

The closer we are to the ideal world, the simpler our SCM is. When our ideal-world intent to work together is thwarted by real-world constraints, we branch one codeline into another and make our changes there. But we don't lose sight of the fact that our changes in the second codeline are really meant for the first. We pull changes from the second codeline into the first as soon as we can.

It is this flow of change between codelines that brings us closer to the ideal world. Each codeline type—mainline, release, and development—has a role in the flow of change:

Mainline

The mainline is the clearing-house for all changes to the software we develop. Whether we submit changes directly to the mainline or integrate changes to it from other codelines, all change eventually reaches the mainline. (see Figure 7-5.)

Figure 7-5. All change flows to the mainline

However, the mainline isn't a free-for-all. It holds the software that's complete enough to enter the release stabilization cycle. So the flow of change to the mainline is tempered by the state of the codelines from which change flows.

Release codelines

Change flows continually from release codelines to the mainline. Every time a bug is fixed in a release codeline, the change that fixed it is integrated into the mainline, as Figure 7-6 shows. This doesn't compromise the mainline, because every change coming from a release codeline has already been reviewed and tested. Moreover, release codeline changes are changes that fix broken things. Thus, merging release codeline changes to the mainline is bound to have a stabilizing effect. It brings the mainline closer to perfection, as is our ideal-world intent. This flow of change continues until the mainline has evolved so much that the bug fixes in a release branch are no longer relevant to it. (In Chapter 9 we'll take a closer look at this.)

Figure 7-6. Bug fixes flowing to the mainline

Release codelines are not normally open to changes from the mainline. For one thing, every change to a release codeline should be a change that stabilizes and finalizes the release. For another, the mainline is changing constantly—it will never be perfect. We don't want the increasing perfection of a release codeline to be sullied by the inherent imperfection of the mainline.

(Although release codelines are not normally open to mainline changes, the unexpected can happen. If we're in the unfortunate position of having to support several releases concurrently, a bug fix in one release may have to be applied to another. That is, we'll have to cherry-pick a change from either a release codeline or the mainline and integrate it into another release codeline. In Chapter 9 we'll cover this in more detail.)

Development codelines

There's also a constant flow of change from the mainline to the development codelines branched from it, as shown in Figure 7-7. In other words, a development codeline is continually updated with changes from its parent codeline. Thus even development codelines benefit from release stabilization. As we fix a bug in a release, we merge the bug fix into the mainline. As the mainline changes, we merge its changes into development codelines.^[*]

^[*] Who is this "we," you ask? See Chapters 8 and 9.

In some cases, bugs can be fixed right in the mainline. Because the mainline is guaranteed to be stable, development codelines can be continually updated with these bug fixes. This gives project teams the benefit of working with the latest, improved code. It also forces them to integrate sooner, rather than later, with development happening outside of their control.

What about the flow of change from development codelines to the mainline? A development codeline can be a hotbed of untested new development. There may be periods of time when a development codeline doesn't build, or when it builds nothing but a basket case. The valve is closed on the flow of change from the development codeline to the mainline during these periods.

Figure 7-7. Flow of change between the mainline and development codelines

But when the development codeline is stable—when new development is complete or at a deliverable state—the valve opens. At these points the development codeline software is delivered to the mainline. Thus change flows from development codelines to the mainline at points of completion. And, because development codelines are always open to mainline changes, other development codelines will receive the completed new development as well.

Development and release codelines can themselves be branched. Quite often they're branched into short-lived, task-specific sub-branches to accommodate unplanned changes. A release codeline can be branched to make a patched version of a released product, for example, and a development codeline can be branched to isolate work on a specific problem or behavior.

7.2.3. Branching from release codelines

In the ideal world, our customers upgrade to our latest release without complaint. In the real world, customers have reasons they can't do that, and we have reasons to keep our customers happy. Reality occasionally puts us the position of having to patch a previously released version. We do this by branching a release codeline into a patch branch.

The flow of change between a patch branch and its release codeline parent is exactly the same as the flow of change between a release codeline and its mainline parent. In other words, the release codeline is continually updated with changes from the patch branch. (Not that there's likely to be much change in the patch branch.) This gives the release codeline the benefit of the patch branch's bug fix. No change flows from the release codeline to the patch branch because the whole point of making the branch was to reproduce and patch an earlier version.

For example, one extremely important customer, still using Release 1.0, finds a critical bug and demands a patch that doesn't require an upgrade to our latest point release, 1.1. To accommodate this customer, we take the 1.0 version of the Release 1 codeline and branch it into a patch branch. We fix the fussy customer's bug in the patch branch and build a new version from a snapshot of the branch. This is the version we give to the customer. (Figure 7-8 shows the patch branch.)

Figure 7-8. A patch branch

The changes we made in patch branch are merged into the Release 1 codeline. This gives the Release 1 codeline the benefit of the patch. Release 1 changes flow into the mainline, of course, bringing the patch with them. (see Figure 7-9).

Figure 7-9. Patch branches and the flow of change

7.2.4. Branching from development codelines

Development codelines branched into sub-branches also inherit the flow-of-change roles of their parents; change flows continually from development codelines to their sub-branches so that work in a sub-branch is always up to date with the parent codeline. Change flows in the other direction only at points of completion. When work in a sub-branch is completed, it's delivered to the development codeline. Thus, each development codeline acts as a mainline for its sub-branches , and each sub-branch behaves like a development codeline.

Consider a team of developers working on an application. They're using a codeline they've named DEVX to develop a major new feature. Two developers plan to help out by overhauling a part of the new feature known as the the Z-widget. It's going to be a ground-up rewrite; the Z-widget won't be working right again for weeks. But a broken Z-widget will make it impossible for other developers to work in the DEVX codeline. (They could simply relax and play ping-pong for two weeks, but that doesn't go over very well in the real world.)

To satisfy constraints of the real world, the DEVX codeline is branched into a codeline named DEVZ. (see Figure 7-10.) The two Z-widget developers complete their overhaul in the DEVZ sub-branch. The rest of the developers continue their work in its parent, DEVX, with an old but stable Z-widget. (And they promise not to touch any of the Z-widget files in DEVX.)

Figure 7-10. A development sub-branch

As changes are made in DEVX, the Z-widget developers pull them immediately into DEVZ. They don't put their changes back into DEVX, however, until their overhaul is done and the new Z-widget is stable. (see Figure 7-11.)

To the Z-widget developers, this satisfies the ideal-world intent to build upon other developers' changes, as if they were working right in DEVX. To the developers working in DEVX, this satisfies the ideal-world expectation that they won't lose project time waiting for broken components to work again.

7.2.5. Soft, medium, and firm: the tofu scale

Every codeline has a characteristic ranking on the "tofu scale." And what is this tofu scale? It's an informal assessment of stability and quality that takes into account:

• How close software is to being released

• How rigorously changes must be reviewed and tested

  Figure 7-11. The flow of change to and from development codelines

  

• How much impact a change has on schedules

• How much a codeline is changing

As shown in (Figure 7-12.), release codelines are highest on the tofu scale; they are "firm." They don't change much, and even the slightest changes to them can impact release schedules because of their rigorous review and testing requirements. The mainline is "medium"—changes do require testing, but release is further out and schedules are more accommodating of them. Development codelines are "soft"—they're changing rapidly, the software in them is farthest from release, and there may not even be tests yet for their newest development.

Figure 7-12. The tofu scale

The flow-of-change rules tell us when file content should be propagated from one codeline to another. The tofu scale tells us how:

• In the firm-to-soft direction, file content can be merged. The target, being softer, is more able than is the donor to absorb the risk and effort of merging.

• In the soft-to-firm direction, file content should be copied. The target is more at risk than the donor in this case. Files should be merged from the firmer codeline to the softer one first, then copied from the softer codeline to the firmer one.

The unwritten contract of collaborative development says that we don't impose unstable changes and we always accept stable changes. The tofu scale gives us a way to tell unstable from stable changes before we impose or accept them.

There's a uniformity to the mainline model that can be described in terms of flow of change and the tofu scale. Aside from the mainline, which is in a category of its own, there are essentially only two codeline types , release codelines and development codelines. No matter how many codelines you have, if you know each codeline's type, you know exactly how and when file content should be propagated between those codelines. This is summarized in Table 7-1; you'll see how this plays out in the chapters that follow.

Table 7-1. How change is propagated

	Release codeline	Development codeline
Tofu rank	Firmer than parent	Softer than parent
Change flows to parent	Continually	At points of completion
Change flows from parent	Never	Continually
File content is propagated	By merging	By copying

7.2.6. A codeline by any other name...

...is still a codeline. If you're saying to yourself that a mainline and a handful of development and release codelines are not going to satisfy your SCM needs, you're right. In fast-paced, large-scale development environments, the fundamental codeline types are adapted and extended to a variety of uses:

Active development streams

Sometimes development projects aren't all that clear—cut. One use of development codelines is to support long-lived, ongoing development work on components. This gives component developers a common, persistent codeline to work in without requiring them to create a new codeline for each task or feature. In Chapter 10, for example, you'll see how a development codeline is used as an active development stream for a GUI component.

Task branches

Task branches are very short-lived codelines branched from either development codelines or release codelines. They can be used to protect release codelines from untested interim changes, or to protect development codelines from destabilizing re-engineering. (The DEVX development codeline described earlier in this chapter in "Branching from development codelines" is a task branch.) In Chapter 9 you'll see how a task branch is used to permit a bug fix to be reviewed and tested before it's introduced into a release codeline.

Staging streams

Staging streams allow you to make extremely frequent releases without having to branch a new codeline for each release. (They're commonly used to support web development. You'll see an example of this in Chapter 11.) A staging stream is essentially a reusable release codeline. Each staging stream is used for a particular stage of release stabilization. Once the stage is completed for a particular release, the codeline is immediately redeployed for the next release.

Private branches, ad hoc branches , and sparse branches

Private branches make it possible for each developer's changes to be reviewed before they are submitted to shared codelines. Private branches can also be used to isolate experimental or proof-of-concept work. Ad hoc branches are created on the fly to give users a place to check in changes they thought they were going to be able to check in elsewhere but found out they couldn't. Sparse branches can be used in any of the aforementioned cases to piggy-back a few changed files onto a full codeline. Examples of all of these will come up in later chapters.

7.2.7. One-way codelines

We also recognize another fundamental codeline type: the "one-way " codeline. One-way codelines house software, but not software development. The following are examples of one-way codelines :

Third-party codelines

Third-party codelines provide a place to store vendor drops—software and source code obtained from external suppliers. Code is typically copied or merged from third-party codelines into development codelines.

Remote depot codelines

In Chapter 6 you read about how you can access depots in other Perforce domains as remote depots . Codelines in remote depots are always one-way codelines—files can be branched, copied, or merged from them, but not into them.

Packaging and distribution streams

Packaging streams can be used to assemble customer-specific configurations of software from released components. Distribution streams can be used to offer released products to customers. (They're the vendor's side of the vendor drop.) Software is delivered from release codelines to packaging and distribution streams; nothing is ever copied or merged in the opposite direction.

We'll revisit one-way codelines in Chapters 9 and 10

7.2.8. Codelines that aren't codelines

Finally, while we're cataloging codeline species, it's worth recognizing that some codelines aren't really codelines at all:

Porting branches

Porting branches contain architecture-specific variants of source code and built objects.

Custom-code branches

Custom-code branches contain source and built objects configured for hardware, customers, locales, and other deployment targets.

Branches like these aren't codelines in their own right. Although it's often difficult to recognize the fact, they're really modules that belong in development codelines, release codelines, and the mainline.^[*]

^[*] For a real-world example of how custom-code branches can become errant codelines, see Changing How You Change (http://www.ravenbrook.com/doc/2003/03/06/changing-how-you-change/), a white paper presented by Peter Jackson and Richard Brooksby at the 2003 Perforce User Conference.

7.2.9. Musings on codeline diagrams

Codeline and flow diagrams help us visualize software evolution. We have all, at one time or another, sat in a room with our colleagues and drawn or pondered diagrams on a whiteboard. Here are some things to keep in mind when you find yourself doing the diagramming:

• When you're drawing a timeline, put release codelines above their parents and development codelines below them. This orders codelines on the tofu scale, with the firmest on the top and the softest on the bottom. (See Figure 7-13.)

  The tofu scale shows the impact of a change at a glance. A change made to a codeline at the top of the diagram, for example, will reach customers soonest, at the greatest risk to quality and scheduling. A change made to a codeline at the bottom of the diagram, on the other hand, doesn't pose a great risk, but it will be a while before it is available to customers.

• Remember the time axis when you're drawing timelines . Plot codeline beginnings and endings in time order. That way, any vertical line you draw will tell you how many active codelines you'll have at that point in time. (See Figure 7-14.)

• If the timelines in a diagram are getting congested, try using slightly parabolic lines instead of horizontal ones (see Figure 7-15s). Parabolic lines also imply divergence—the greater the vertical distance to the mainline, the more the codeline is likely to have diverged from the mainline.

  Figure 7-13. Ordering codelines on the tofu scale

  

  Figure 7-14. Ordering codeline beginnings and endings on the time axis

  

• Remember that historical flow of change isn't the same as intended flow of change. Use flow diagrams in addition to timelines to help people understand how change is intended to flow between codelines. Show the tofu scale in flow diagrams as well, as shown in the example in Figure 7-16.

• Finally, recognize that if your codeline diagram is complicated, your SCM process is going to be complicated. Simplifying the diagram may be a good first step in reducing the complexity of your SCM plan.

Figure 7-15. Using parabolic lines to reduce congestion in a diagram

Figure 7-16. A flow diagram