Frame Pool Effect -- A UWP Custom Video Effect

To use the frame pool effect one allocates memory in the amount of  8 + (number of pool frames) * (frame size in pixels * 4). The allocated (Marshall.AllocHGlobal eg) IntPtr is then passed as the property "FarPtr" on startup.

A six-frame sized pool at 1920 x 1080 plus 8 bytes for the frame write position pointers, which is written by the effect to the front of the memory block as an int32s, is 1080 x 1920 x 4 x 6 + 8 =  49,766,408 bytes.

One also writes the length of the video frame in bytes (property "ByteLength"), and the number of pool frames (property "PoolSize"). It is up to the user to make sure all these items are correctly calculated and allocated before initializing the effect. PoolSize * ByteLength + 8 has to be the memory allocation size minimum.

C-Sharp specific: read the frame to managed memory:

FarPtr is the pointer that was allocated. ByteLength bytes in frame. 

byte[] bt2 = new byte[ByteLength];
int ofs = Marshal.ReadInt32(FarPtr);  
IntPtr FarPtr2 = IntPtr.Add(FarPtr, ofs);
Marshal.Copy(FarPtr2, bt2, 0, bt2.length);

VB specific: Read the frame to managed memory

Dim bt2(ByteLength - 1) as Byte
Dim ofs as Integer = Marshal.Readint32(FarPtr)
Dim FarPtr2 as IntPtr = IntPtr.Add(FarPtr, ofs) 
Marshal.Copy(FarPtr2, bt2, 0, bt2.length)

The integer at address FarPtr+4 contains the offset of the frame being written, or just having been written. The idea of this Frame Pool Effect is to always have a safe-to-read current frame immediately available.

Code:

public sealed class FramePoolEffect : IBasicVideoEffect
    {
        private IntPtr remotePointer;
        private bool skipWrite;
        private int byteLength;
        private int poolSize;
        
        public bool TimeIndependent { get { return true; } }

        public MediaMemoryTypes SupportedMemoryTypes { get { return MediaMemoryTypes.Gpu; } }

        public bool IsReadOnly { get { return true; } }

        private CanvasDevice canvasDevice;

        public IReadOnlyList<VideoEncodingProperties> SupportedEncodingProperties
        {
            get
            {
                var encodingProperties = new VideoEncodingProperties();
                encodingProperties.Subtype = "ARGB32";
                return new List<VideoEncodingProperties>() { encodingProperties };
            }
        }

        public void SetEncodingProperties(VideoEncodingProperties encodingProperties, IDirect3DDevice device)
        {
            canvasDevice = CanvasDevice.CreateFromDirect3D11Device(device);
        }

        public void SetProperties(IPropertySet configuration)
        {
            object brb;
            if (configuration.TryGetValue("SkipWrite", out brb))
            {
                skipWrite = (bool)brb;
            }
            else
            {
                skipWrite = false;
            }
            object brb2;
            if (configuration.TryGetValue("FarPtr", out brb2))
            {
                remotePointer = (IntPtr)brb2;
            }
            else
            {
                remotePointer = IntPtr.Zero;
            }
            object brb3;
            if (configuration.TryGetValue("ByteLength", out brb3))
            {
                byteLength = (int)brb3;
            }
            else
            {
                byteLength = 0;
            }
            object brb4;
            if (configuration.TryGetValue("PoolSize", out brb4))
            {
                poolSize = (int)brb4;
            }
            else
            {
                poolSize = 0;
            }
            if (remotePointer == IntPtr.Zero | poolSize == 0 | byteLength == 0)
            {
                skipWrite = true;
            }
        }

        public void DiscardQueuedFrames()
        {

        }

        public void Close(MediaEffectClosedReason reason)
        {
            //memory is handled by main program
        }

        public void ProcessFrame(ProcessVideoFrameContext context)
        {
            using (CanvasBitmap inputBitmap = CanvasBitmap.CreateFromDirect3D11Surface(canvasDevice, context.InputFrame.Direct3DSurface))
            {
                if (!skipWrite)
                {
                    long now = DateTime.Now.Ticks;
                    int frameOfs;

                    frameOfs = Marshal.ReadInt32(remotePointer, 4);
                    if (frameOfs != 0)
                    {
                        Marshal.WriteInt32(remotePointer, frameOfs); //'last write position is read position'
                        frameOfs -= 8;
                        frameOfs /= byteLength; //'baseframe number of last write'
                        frameOfs = ((frameOfs + 1) % poolSize) * byteLength + 8;
                        Marshal.WriteInt32(remotePointer, 4, frameOfs);
                    }
                    else
                    {
                        frameOfs = 8;
                        Marshal.WriteInt32(remotePointer, 4, frameOfs);
                        Marshal.WriteInt32(remotePointer, (int)-1); //'pool not ready flag'
                    }
                    byte[] bt2 = CanvasBitmap.CreateFromDirect3D11Surface(canvasDevice, context.InputFrame.Direct3DSurface).GetPixelBytes();
                    Marshal.Copy(bt2, 0, IntPtr.Add(remotePointer, frameOfs), bt2.Length);
                    Debug.WriteLine((DateTime.Now.Ticks - now) / 10000);
                }
            }
        }
    } 

Note about code listing. When copied the right arrow (<) and left arrow (>) may not show up properly, and instead be represented by the string literals (&lt;) and (&gt;) respectively. Replace those four character literals with the correct arrow in that case. 

Disclaimer: This is free for personal use only. Place of origin, California, US.