Take the 2-minute tour ×
Personal Productivity Stack Exchange is a question and answer site for people wanting to improve their personal productivity. It's 100% free, no registration required.

My question: is there a reliable and simple version control system/mechanism/platform/... for projects with content that is effectively monolithic, opaque stuff?

Most of the projects I work on contain a mishmash of the following file types:

  1. Plain-text content (C, Java, ..., data files, bash scripts, ...)
  2. MS Office documents (Word, Excel, Visio, ...)
  3. Vector graphics files (SVG, EPS, ...)
  4. Simulink® model files (.mdl, similar in format to JSON)
  5. Our custom-made unit test framework (generated XML files)
  6. ...misc. other things with similar limitations

Currently we always put everything in one big subversion repository. Obviously, this only works well for the plain-text files. All the rest is unmergeable when treated as plain text.

For some of them there are "managed" version control systems, which we indeed use on top of subversion. For example, for MS Word documents we use the "track changes" feature. This has effectively become our version control system specific for MS Word documents. We then create copies of the document in the subversion repository whenever some version of the document needs to be tagged/delivered.

Similarly, there are managed ways to concurrently develop Simulink models, concurrently author vector graphics, etc.

But as you can imagine, I'm quite unhappy up with this workflow. Something like "track changes" doesn't prevent two users from editing the same document at the same time. Changes of both cannot be merged, and merging needs to be done manually (I'm not going to even mention document corruption). Same with the Simulink framework; real concurrency is impossible.

It's just a pain this, not to mention completely inefficient since a lot of manual copying and twiddling of files is required to keep everything properly versioned -- having everything in a subversion repository has become not much more than a convenient alternative to e-mailing documents around (which is only convenient for the people not managing the repository...).

Something like Sharepoint could solve the problem for MS Word documents, but it's not a general solution; we still have these other file types to deal with. Something like subversion plugins/hooks/etc. could work, but can be hard to get right and also a pain to keep up to date, plus it could be a significant investment since we'd probably need to develop custom plugins for things like Simulink model files...

So my question: is there a reliable and simple version control system/mechanism/platform/... for projects with content that is effectively monolithic, opaque stuff?

share|improve this question
1  
As this solves a particular problem (set), I don't see it as being off-topic or looking for a list. +1 as I'm interested. –  Raystafarian Feb 7 at 14:51
    
very interesting question! Could you describe more, why MS Sharepoint is not an option for your non-MS-Office-document types? –  Martin Feb 8 at 8:13
1  
@Martin: Working with plain text documents in subversion/git allows multiple users to work on a single document in parallel. Changes by each user are merged automatically on commit. If all users commit often, conflicts are rare, even when all people worked on the same section at the same time. Perhaps I don't know Sharepoint well enough, but as I understood it, Sharepoint is capable of managing documents and facilitating collaboration, but only through a mechanism akin to locking; that is, when someone edits a document, no-one else can edit that document...Collaboration is serial. –  Rody Oldenhuis Feb 10 at 7:51
1  
@Martin: In Sharepoint, AFAIK, only for MS office documents can collaboration be parallel. For those other things (for my work, the most important output of all projects), lockins is used. Correct me if I'm wrong though :) –  Rody Oldenhuis Feb 10 at 7:54
add comment

2 Answers 2

I would avoid file type specific version control systems as having to look in too many places would not help your situation.

For that reason I would recommend moving from SVN to Git for the repository for a few reasons. I know that would not be an easy task but keep reading for my reasoning.

  1. Git treats all file types the same (text, binary, etc.)
  2. Content is compressed with zlib to reduce its size (repo size can further be reduced with packfiles)
  3. File history is maintained outside of any file type specific tools (like MS Word) and can be traced back to a git user and commit.

Let me expand on some of that a little bit.

  1. Git doesn't care if a file is text, binary, image, etc. Everything is treated the same and it doesn't store only differentials of files like SVN does. Each commit will store the entire file contents and can easily be reverted if something breaks (without merging diffs). You obviously have the powerful Git tools like cherry picking files or checking out tags/commits to resent an entire directory.

  2. Git automatically will zlib compress files where it can (text will benefit the most from this) but for binary and image files this obviously won't be very beneficial. Git will also remove duplicate files and store differentials (similar to SVN) if you push your commit to a remote server or run git gc on the repo. This would greatly reduce storage requirements and repo cloning time for people that don't pull changes regularly. You can see this explanation of packfiles for more information.

  3. Because your repo is so diverse it is important to not have to look to multiple revision systems for tracking changes. You want the tracking to happen as much at the "storage layer" as possible and have one interface to see all changes across the repo. SVN does that for you currently you seem to be having problems with conflicts and file management. I am not very experienced with SVN but from what I read from other people who are much smarter than I am, Git does a better job at merging/branching so those problems shouldn't be as drastic.

With Git your workflow might be different than it currently is but you should be able to improve many aspects with branching and local repositories. For a generic git workflow you could check out Atlassian's tutorial which might.

Alternatively, you could use something like git-svn to interface git with your current SVN server but I don't think that would have the merging and storage benefits you are looking for.

There's also git-bigfiles which I have heard of but never used so you may want to try it out if your repository files become too big.

share|improve this answer
    
very interesting! Does Mercurial offer the same advantages compared to SVN, or is there something really special to git in your points? –  Martin Feb 11 at 6:17
1  
Mercurial would allow tracking changes at the file system layer (as would SVN) but I am not familiar enough with how it stores files to be able to say for sure with mixed content repos. The FAQ points out that it will store full files or diffs (similar to git) but it's up to hg to choose when to do each. Thanks to your question I plan on reading this ebook later today to find out more hgbook.red-bean.com –  rothgar Feb 11 at 15:18
    
thanks for your effort! I'm curious to learn more about the differences between hg and git in this context. –  Martin Feb 12 at 1:42
    
Make sure that git and your git tools handle binary files. A diff on binary is a slow thing, and makes no sense. More on sharp corners of git+binary here: stackoverflow.com/questions/4697216/is-git-good-with-binary –  Steed Feb 17 at 8:53
    
This is defintately the best answer. git is much much better than Mercural and Mercurial is infinitely better than SVN. –  mrsteve Mar 1 at 10:41
show 1 more comment

Dropbox with the Packrat feature (has to be paid extra) turned on will keep an unlimited number of previous versions of any files you put in it.

description taken from the Dropbox Homepage:

Packrat is a feature that gives you unlimited deletion recovery and version history.

By default, Dropbox saves a history of all deleted and earlier versions of files for 30 days for all Dropbox accounts. If you purchase the Packrat add-on for your account, Dropbox will save your files for as long as you have the feature, so you won't have to worry about losing a deleted folder or file.

Dropbox also saves and indicates "conflicted copies" so you can see if two people edit/save the same file at the same time.

share|improve this answer
    
can Packrat also export or restore the state of the whole "repository" at a certain point of time (e. g. state of all files "last tuesday on 13PM"), or does it just keep the history of each single file –  Martin Feb 8 at 11:02
1  
@Martin: hmm the documentation is a bit sparse, but from what I gather, Packrat is not much more than an unlimited version of what you also get with a normal Dropbox account. That is, full change and deletion history, and undo capability, on a per-file and per-folder basis. –  Rody Oldenhuis Feb 10 at 7:58
    
I like the idea! I've been an avid Dropbox user for the past two years myself :) Do you happen to know if checking out previous versions of the repository is possible via command line? –  Rody Oldenhuis Feb 10 at 8:00
    
But it doesn't quite take away the inability to merge changes...I mean, if two or more users edit an MS word document at the same time, those changes cannot be merged... –  Rody Oldenhuis Feb 10 at 8:03
    
Microsoft Word has the ability to combine two documents with the "Combine" option if you do discover these disparate edits. –  Marina Martin Feb 10 at 13:38
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.