Introduction

I'm a big fan of continuous integration, small checkins and very quick checkins. It's easier to debug afterwards and trying to puzzle together from perforce history afterwards is much easier when the changes are small and atomic. Now, that's what I like to do. Sometimes things don't really work out in the real world like you want it to so the other day I found myself with the fun situation of having two weeks worth of changes to try to integrate back into the main line. Of course nothing was backed up, nor did I feel particularly worried about it at the time. I had however done a lot of work, some of the changes involved the fruits of heavy thinking and other changes were the result of an hour or two of menial changes to the code that was a result of some other change.

So I'm pretty happy to integrate everything and merge, it goes pretty well. I'm almost at the point where I can check in when I notice that there is another overlapping change that just got committed to the repository. I get that one and notice that I need to resolve one of the .vcproj files I have checked out for edit. During the merge I realize that everything I've done to the file really doesn't make any sense anymore, just old vestiges of experiments several days old. So I quit the merge and simply revert the file. I can also mention that I do all this in p4win. I switch back to Visual Studio and I'm greeted by this dialog box:

Hm. Well, undo checkout seems reasonable since that's what I just did. Stupid Visual Studio just didn't catch what happened in the other program. Click. Compile. Errors! WTF? I'm digging through a lot of errors that really doesn't make any sense... then it dawns on me. Undo checkout reverts all the files bound to the project! The horror! At this point I go "fuck fuck fuck" like a mantra and occasionally groan a little. Not pretty.[1]

In the end I had to reconstitute much of the files from memory, I had a little bit of help from one of the caches from our builds. No undelete tool caught anything (we even downloaded a couple off the Internet and tried). The apple was very sour at this point.

Eagle eye hindsight

Peter Paranoid usually backups his source files to a network drive every hour. This would of course have saved me from all this pain. The problem with this approach is that you have to remember to do it. Crap. Another option would have been to not trust any dialogs in Visual Studio related to source control (it simply can not do any of it correctly).

Speaking of backing up your source every now and then, there is no easy tool to do this out of the box in perforce. In many ways I feel that perforce is much like Maya, they provide a bit of basic orthogonal functionality and then tell you to roll your own integration into your workflow. There are a lot of scripts that do this and that out there, some of them are hidden inside the public perforce depot. Others, just reside on webpages like this. I've written similar scripts in the past, most of which are hidden in other (company) repositories. I would have really benefited from one at work now, but I've never felt the motivation to write one at work quite yet. This even pushed me over the line though, so I spent some time writing a solution that hopefully works for me. Too lazy to roll into work, I just installed a local version of the perforce server (now that I finally got rid of Vista so that I can run the client software).

The script

As the returning reader might have guessed, this is just all leading up to yet another python script. This might even be useful! I hope so at least. The idea is that we can record the information from perforce about our clientstate and save this off in a separate file, maybe somewhere on the network somewhere so that we can have a backup. The script can also double as a transfer media between different clientspecs, e.g. you want to send changes you have locally to another developer but you don't want to commit it to the repository (think buildserver here, you can send this zip file to a buildserver, it verifies that the changes work and then tries to apply them to the repository itself, or just give the go ahead).

I might also point out that this is a python 2.5 script. The current migration to 3.0 is underway and for people that try to use this on Py3k as it's called, it probably won't work. Another caveat is that I just think this works. Please be careful when you do this on live repositories and changelists. While I've tried out the script myself, there are no guarantees that it will work for you since perforce is so flexible and you can have all kinds of whacky configurations and little gottchas that I've never seen that might do horrible things to your hard drive. Best option is to carefully look through the script and determine if it will work for your site.

The script operates in two major modes, compress and extract. The default mode is extract. The basic idea is to save all the opened files into a zip archive that we later can extract to any client path. In addition to the actual files we also save some meta information about the kinds of operations that were made, i.e. delete, add, branch or edit. There are quite a lot of command line switches, they should be pretty self explanatory for the experienced perforce user.

It's suggested to hook it up as a custom tool in p4win, at which point you can easily backup any given changelist simply by right clicking on the changelist and select the backup tool. Go to Tools->Customize and add the following tool (of course changing the paths to whatever you prefer on your system):

That should pretty much give you a context sensitive menu like this:

Thoughts and improvements

The script doesn't handle stale files in your working directory very well, it follows the default perforce behavior to not clobber writeable files in order to err on the side of the caution since it tries very hard not to delete any data from your drive that is not under perforce control. I decided to keep this behavior but it might prove too cumbersome in the wild. Time will tell if it was a good choice.

The script also contains some code for inserting timestamps and descriptions into the archive files, this is mainly for a future application of mine that I was thinking about; a separate browser for quickly scanning through a list of archives and choosing the correct one to unshelf. For this to work you might want to see the descriptions and the date inside the archive.

In closing

I've just recently come to use the perforce integration into visual studio, I used it for a brief while many years ago and after a few painful days I wrote a script similar to badger config to let us generate two sets of visual studio project files and solution, one for us that didn't want to run the visual studio plugin and one for the people who insisted that it was the best thing since sliced bread. I'm back in the trenches with the SCC plugin, and this was my first really painful experience with it (other than the horrible startup times and reload times). Well, I guess I'm not clicking on that dialog before backing my stuff up again.

There are several alternatives to running the source control plugin in visual studio, many have written plugins that are faster and easier to use to integrate with perforce (see here and here for example). In the meanwhile I can just be very wary of the dialogs...

---

2008-01-31: Update

I've uploaded two small patches to the script. One for handling multi line comments and the second for handling adding new files to the repository. Seems that the last part broke sometime during the development.

2008-05-26: Update

I've started a new open source project for all the perforce stuff I've written. Ironically it is hosted on a subversion server. You can see it over here: http://code.google.com/p/p4scripts.

Resources

• Practical Perforce, a pragmatic take on how to manage and do common operations in perforce the way it was intended. Go with the flow and read it!
• Download the script.
• Another tool that's also called p4shelf. Also some stuff in the public perforce depot (see wiki here).
• p4tar/p4untar, Jeff Grills' perl scripts (another link here).

Footnotes

• [1] What I should have done of course (now in 20/20 hindsight) was to click continue. Or even better, simply kill visual studio as soon as it tries to do something related to source control.

Comments