iOS Video Quality

Resolution, framerate, and quality of video captured using iOS devices

This is a brief overview of my findings while working on a recent project at Hudl on the Ubersense app.

Which resolutions are available on an iOS device?

This depends on what model device you have, and whether you're talking about the front or back camera. Using Apple APIs, you can get a list of all the AVCaptureDeviceFormats for a camera -- here's a code snippet doing just that:

AVCaptureDevice *camera = ...;
for (AVCaptureDeviceFormat *format in [camera formats])
{
    CMVideoDimensions dims = CMVideoFormatDescriptionGetDimensions([format formatDescription]);
    CGFloat fps = 0;
    if ([format.videoSupportedFrameRateRanges count] > 0)
    {
        AVFrameRateRange *range = format.videoSupportedFrameRateRanges[0];
        fps = range.maxFrameRate;
    }
    NSString *title = [NSString stringWithFormat:@"%@ x %@ @ %@", @(dims.width), @(dims.height), @(fps)];
    NSLog(@"%@", title);
}

Let's see what's available on the iPhone6 back camera:

• 192x144, 30 FPS
• 352x288, 30 FPS
• 480x360, 30 FPS
• 640x480, 30 FPS
• 960x540, 30 FPS
• 1280x720, 30 FPS
• 1280x720, 240 FPS
• 1920x1080, 30 FPS
• 1920x1080, 60 FPS
• 2592x1936, 30 FPS
• 3264x2448, 30 FPS

And the iPhone6 front camera:

• 192x144, 60 FPS
• 352x288, 60 FPS
• 480x360, 60 FPS
• 640x480, 60 FPS
• 960x540, 60 FPS
• 1280x720, 60 FPS
• 1280x960, 60 FPS

Examples

What differences can you detect between these 5 different AVCaptureDeviceFormats?

I recorded 5 videos, each with a different AVCaptureDeviceFormat in Ubersense at 3.5x zoom. I then opened the videos in the Ubersense player, zoomed in again to 3.5x, and took a screenshot.

(Which formats I used are at the end of this post.)

Conclusions

On iOS devices, you can't use resolution as a proxy for quality. In other words, the naive assumption that the higher the resolution, the better the quality; and the lower the resolution, the worse the quality is false.

Quality depends on more than just resolution. It's a good idea to do some tests using real data.

Format used

The formats I used, in order, were:

• 480x360, 30 FPS
• 1280x720, 30 FPS
• 1280x720, 240 FPS
• 1920x1080, 30 FPS
• 1920x1080, 60 FPS

The most obvious difference is that the 240 FPS format looks way worse than everything else; presumably a tradeoff of such a high frame rate. Otherwise, I find it difficult to choose between the various formats -- it's suprising to me that 1920x1080 isn't clearly superior to 640x480, given the gigantic difference in resolution.

What do you think?

Make git merges easier, safer, and more transparent

Merges between semantically diverged branches are difficult to resolve

Lately at work, I've had to merge git branches that have significantly diverged since their common ancestor. When I run a "git merge" command, I end up with many conflicts that I have to solve. Since I couldn't be sure I was solving them correctly, I came up with a strategy for handling difficult merges that makes it easy to see exactly what the merge conflicts were, and how I resolved them.

Briefly, the strategy is:

• create a new branch pointing to the same commit as your target branch, and run "git merge source-branch" (the purpose of creating a new branch specifically for resolving the merge is to help prevent accidentally screwing up your branches)
• there will be conflicts; simply add all the files with conflicts as-is and commit (I prefer not to change the default commit message)
• in subsequent commits, manually resolve all the conflicted files
• push the new resolution branch up to github and open a PR against the target branch, allowing your team members to review the merge conflicts and resolution (team members can now see the merge conflicts and my resolution, and can comment on it on github)
• after addressing any comments, merge the PR (this is now a fast-forward merge, if no changes have been made to the github repo's target-branch in the meantime)

Example

You can find a complete git repo of this example here.

The git repo has a single file, "a.txt". Two branches, named "target-branch" and "source-branch", introduce conflicting changes to the same line of "a.txt".

Attempting to merge these branches results in conflicts.

Here are the contents of "a.txt" in the common ancestor (517537):

1
2
3

"a.txt" in the target branch (7ac885):

1
1.4
2
3

and "a.txt" in the source branch (d65a7e):

1
1.6
2
3

As you can see, the two versions of "a.txt" in the target and source branches differ at line 2.

I'm going to create a new branch "resolution-branch" (which initially points to the same commit as "target-branch"), and merge "source-branch" into "target-branch":

$ git checkout target-branch
Switched to branch 'target-branch'

$ git checkout -b resolution-branch
Switched to a new branch 'resolution-branch'

$ git merge source-branch
Auto-merging a.txt
CONFLICT (content): Merge conflict in a.txt
Automatic merge failed; fix conflicts and then commit the result.

$ git status
On branch resolution-branch
You have unmerged paths.
  (fix conflicts and run "git commit")

Unmerged paths:
  (use "git add ..." to mark resolution)

 both modified:   a.txt

no changes added to commit (use "git add" and/or "git commit -a")

$ git diff
diff --cc a.txt
index a41e2e2,26a22ad..0000000
--- a/a.txt
+++ b/a.txt
@@@ -1,4 -1,4 +1,8 @@@
  1
++<<<<<<< HEAD
 +1.4
++=======
+ 1.6
++>>>>>>> source-branch
  2
  3

$ git add a.txt

$ git commit
[resolution-branch f621093] Merge branch 'source-branch' into resolution-branch

I then manually resolve "a.txt", so that the diff looks like this:

$ git diff
diff --git a/a.txt b/a.txt
index ce6567d..2357f17 100644
--- a/a.txt
+++ b/a.txt
@@ -1,8 +1,4 @@
 1
-<<<<<<< HEAD
-1.4
-=======
-1.6
->>>>>>> source-branch
+1.5
 2
 3

and add and commit "a.txt":

$ git add a.txt 

$ git commit -m "resolve a.txt: compromise on 1.5"
[resolution-branch e244fea] resolve a.txt: compromise on 1.5
 1 file changed, 1 insertion(+), 5 deletions(-)

this is what the commit graph now looks like on "resolution-branch":

$ git log --graph --all
* commit e244fea9d08a3a8351b954c6ab0e71622087eae8
| Author: Matt Fenwick <...>
| Date:   Mon May 11 12:26:39 2015 -0500
| 
|     resolve a.txt: compromise on 1.5
|    
*   commit f62109387f047b16b1b0591067550e57813b8164
|\  Merge: 7ac885c d65a7ec
| | Author: Matt Fenwick <...>
| | Date:   Mon May 11 12:22:48 2015 -0500
| | 
| |     Merge branch 'source-branch' into resolution-branch
| |     
| |     Conflicts:
| |             a.txt
| |   
| * commit d65a7ecab55155ee8506a3a676f967bb1e16d87c        <-- this is "source-branch"
| | Author: Matt Fenwick <...>
| | Date:   Mon May 11 12:22:04 2015 -0500
| | 
| |     1.6
| |   
* | commit 7ac885c2f463ea24a5d40239a301d6cc4d1a421e        <-- this is "target-branch"
|/  Author: Matt Fenwick <...>
|   Date:   Mon May 11 12:21:08 2015 -0500
|   
|       1.4
|  
* commit 517537fd892e39c68917eeaf012b5ebe2c1bc374          <-- this is "common-ancestor-branch"
  Author: Matt Fenwick <...>
  Date:   Mon May 11 12:20:26 2015 -0500
  
      1,2,3

Finally, I open a pull request for merging "resolution-branch" into "target-branch". Merging this PR is a fast-forward (and thus, trivial) since there have been no updates to "target-branch" in the meantime.

You can find a complete example of this here.