Marshal UTF8 Strings in .NET

Wow, what a pain in the butt. .NET strings are stored internally as UTF16, not UTF8, so if you‘re marshaling strings to and from a library that wants strings as UTF8, you have to manually marshal them yourself.

This took me a whole day to figure out why my my .NET wrapper library wasn‘t working, and a whole other day to figure out how to work around it and debug the code. If this code saves at least one person the amount of time I lost then I‘m satisfied.

        public class MarshalPtrToUtf8 : ICustomMarshaler
        {
            static MarshalPtrToUtf8 marshaler = new MarshalPtrToUtf8();

            public void CleanUpManagedData(object ManagedObj)
            {

            }

            public void CleanUpNativeData(IntPtr pNativeData)
            {
                Marshal.Release(pNativeData);
            }

            public int GetNativeDataSize()
            {
                return Marshal.SizeOf(typeof(byte));
            }

            public int GetNativeDataSize(IntPtr ptr)
            {
                int size = 0;
                for (size = 0; Marshal.ReadByte(ptr, size) > 0; size++)
                    ;
                return size;
            }

            public IntPtr MarshalManagedToNative(object ManagedObj)
            {
                if (ManagedObj == null)
                    return IntPtr.Zero;
                if (ManagedObj.GetType() != typeof(string))
                    throw new ArgumentException("ManagedObj", "Can only marshal type of System.String");
                byte[] array = Encoding.UTF8.GetBytes((string)ManagedObj);
                int size = Marshal.SizeOf(array[0]) * array.Length + Marshal.SizeOf(array[0]);
                IntPtr ptr = Marshal.AllocHGlobal(size);
                Marshal.Copy(array, 0, ptr, array.Length);
                Marshal.WriteByte(ptr, size - 1, 0);
                return ptr;
            }

            public object MarshalNativeToManaged(IntPtr pNativeData)
            {
                if (pNativeData == IntPtr.Zero)
                    return null;
                int size = GetNativeDataSize(pNativeData);
                byte[] array = new byte[size - 1];
                Marshal.Copy(pNativeData, array, 0, size - 1);
                return Encoding.UTF8.GetString(array);
            }

            public static ICustomMarshaler GetInstance(string cookie)
            {
                return marshaler;
            }
        }

You‘ll notice that there‘s a lot of data copying going on and there are a few copies of string made. Yep, that‘s because the .NET framework can‘t just pin the array in memory that stores the string (remember, strings are stored as UTF16 in the .NET framework) and you have to make the conversion yourself.