Slower MJPEG decoding using video memory compared to system memory

Media SDK Client or Media Server Studio version installed: Intel Media SDK 2016 R2

Processor Type: Intel i7 6700 with 16 GB of RAM on a Gigabyte Gaming 7 Z170X motherboard

Driver Version: 15.45.10.4542

Operating System: Windows 10 Pro

Media SDK System Analyzer: I have attached the output from the system analyzer.

Concise Description of the Issue:

I have been using the Intel Media SDK 2016 R2 to get a MJPEG decoder working.
I followed the media SDK tutorials code as well as the media SDK samples and got an implementation using system memory up and running.

For my application I need maximum performance thus I now need to get the decoder working using video memory. I have it decoding and producing
valid frames while using video memory, however, it is much slower than it should be... in fact currently using the system memory approach is 3 times faster than when using video memory so clearly something is wrong with my code. I get about 100fps using system memory and only 30 fps when using video memory.

I've looked through other forum posts (there aren't many) and can't see what I am doing wrong.
I've looked through the tracer logs for any warnings about partial acceleration but there aren't any. I have also tried it on another machine (a laptop with an i7 3630QM with integrated HD4400 graphics) and saw the same result, system memory decoding much faster than video memory.

Attached with this post are tracer logs when using the system memory approach and another using video memory.
I have also included the relevant code (I'm trying to have a minimal implementation, some of the functions here directly use some of the functionality 
in the MediaSDK tutorials with files such as common_directx11, common_utils, common_utils_windows).
Please let me know if you need anything else in order to help me with this problem. I suspect I am missing something simple but I can't see it.

Your help in this would be greatly appreciated!