Posted onSep 9, 2018 • Originally published atrendered-obsolete.github.io onSep 9, 2018

Loading Native Libraries in C#

While possible for C# to call functions in native libraries, you must avoid mixing 32 and 64-bit binaries at runtime lest you invite the wrath ofBadImageFormatException. Woe unto you.

Different hardware and OSes have different caveats. This is mostly talking about 64-bit Windows 10 and occasionally OSX (64 bit).

“Any CPU”

.Net assemblies can be compiled with a platform target of “Any CPU”.This SO answer covers it nicely:

Binary targetting “Any CPU” willJIT to “any” architecture (x86, x64, ARM, etc.)- but only one at a time.
On 64-bit Windows 10, an “Any CPU” binary will JIT to x64, and it can only load x64 native DLLs.

What happens if you try to load a 32-bit assembly into a 64-bit process (or vice-versa)?BadImageFormatException.

Windows “Hack”

Pinvoke is one approach to call functions in native DLLs from C#.

For several years I’ve used a well-known trick to selectively load 32/64-bit native libraries in Windows desktop applications:

classADLWrapper{[DllImport("LibADLs")]staticexternintLibADLs_GetAdapterIndex(IntPtrptr);staticADLWrapper(){// If 64-bit process, need to load 64-bit native dll. Otherwise, 32-bit dll.// Both dlls need to have same filename and be dllexport'ing the same functions.if(System.Environment.Is64BitProcess){varhandle=LoadLibraryEx(pathTo64bitLibs+"LibADLs.dll",IntPtr.Zero,0);if(handle!=IntPtr.Zero){//...}}else{// Load 32-bit dll}}}

This assumes you have two identically named shared libraries with different paths. For example,x86/libadls.dll andx64/libadls.dll. It’s slight abuse ofthe way dynamic libraries are found. In this case a snippet of C# wrapper around a support library for AMD’s Display Library (ADL).

Prior to the introduction ofIs64BitProcess in .Net 4.0, the value of(IntPtr.Size == 8) could be used instead.

When dealing with dynamic libraries with different names (as is the case with ADL), because the argument toDllImportAttribute must be a constant we have the unflattering:

interfaceILibADL{Int64GetSerialNumber(intadapter);}classLibADL64:ILibADL{[DllImport("atiadlxx.dll")]// 64-bit dllstaticexternintADL_Adapter_SerialNumber_Get(intadapter,outInt64serial);publicInt64GetSerialNumber(intadapter){Int64serial;if(ADL_Adapter_SerialNumber_Get(adapter,outserial)==0){returnserial;}return-1;}}classLibADL32:ILibADL{[DllImport("atiadlxy.dll")]// 32-bit dllstaticexternintADL_Adapter_SerialNumber_Get(intadapter,outInt64serial);//...

And then somewhere else:

if(Environment.Is64BitProcess){adl=newLibADL64();}else{adl=newLibADL32();}

Simple and effective. But doesn’t work on OSX (or Linux).

GetDelegateForFunctionPointer

Another approach I first came across when experimenting withNNanomsg usesGetDelegateForFunctionPointer():

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]publicdelegateintnn_socket_delegate(intdomain,intprotocol);publicstaticnn_socket_delegatenn_socket;staticvoidInitializeDelegates(IntPtrnanomsgLibAddr,NanomsgLibraryLoader.SymbolLookupDelegatelookup){nn_socket=(nn_socket_delegate)Marshal.GetDelegateForFunctionPointer(lookup(nanomsgLibAddr,"nn_socket"),typeof(nn_socket_delegate));}

Wherelookup isGetProcAddress() on Windows (anddlsym() on posix platforms).

A similar approach isused by gRPC.

Ok, but unwieldly forlarge APIs like nng (which is where we’re headed).

SWIG et al.

It’s hard to talk about interfacing with native code and not come acrossSWIG (or other similar technologies).

Over the last 15 years we’ve crossed paths a few times. The first time being when tasked with creatinga python 1.5 wrapper for a Linux virtual device driver. Most recently,a half-hearted attempt to use it with AllJoyn.

But it always ends the same way: frustration trying to debug my (miss-)use of a non-trivial tool, and befuddlement with the resulting robo-code.

“Modern” Nupkg

Necessity being the mother of invention,supporting multiple platforms begatadvances in nuget packaging to address the issue.

We’ve been trying to get acsnng nupkg containing a native library (nng) working on OSX:

Build the dynamic library (libnng.dylib):mkdir build && cd build && cmake -G Ninja -DBUILD_SHARED_LIBS=ON .. && ninja
Copy intoruntimes/osx-x64/native of the nupkg

InRepSocket.cs:

[DllImport("nng",EntryPoint="nng_rep0_open",CallingConvention=Cdecl)][return:MarshalAs(I4)]privatestaticexternint__Open(refuintsid);

Which works fine on Windows, but on OSX:

Mono: DllImport error loading library 'libnng': 'dlopen(libnng, 9): image not found'.Mono: DllImport error loading library 'libnng.dylib': 'dlopen(libnng.dylib, 9): image not found'.Mono: DllImport error loading library 'libnng.so': 'dlopen(libnng.so, 9): image not found'.Mono: DllImport error loading library 'libnng.bundle': 'dlopen(libnng.bundle, 9): image not found'.Mono: DllImport error loading library 'nng': 'dlopen(nng, 9): image not found'.Mono: DllImport error loading library '/Users/jake/test/bin/Debug/libnng': 'dlopen(/Users/jake/test/bin/Debug/libnng, 9): image not found'.Mono: DllImport error loading library '/Users/jake/test/bin/Debug/libnng.dylib': 'dlopen(/Users/jake/test/bin/Debug/libnng.dylib, 9): image not found'.Mono: DllImport error loading library '/Users/jake/test/bin/Debug/libnng.so': 'dlopen(/Users/jake/test/bin/Debug/libnng.so, 9): image not found'....Mono: DllImport error loading library '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/libnng.dylib': 'dlopen(/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/libnng.dylib, 9): image not found'....

The additional library loading information is enabled by setting environment variables:

# .NET FrameworkMONO_LOG_MASK=dllMONO_LOG_LEVEL=info# .NET CoreDYLD_PRINT_LIBRARIES=YES

In Visual Studio for Mac: Right-click projectOptions->Run.Configurations.Default under “Environment Variables”
Visual Studio Code: in"configurations" section of.vscode/launch.json add"env": { "DYLD_PRINT_LIBRARIES":"YES" }

One solution comes fromUsing Native Libraries in ASP.NET 5 blog:

Preload the dylib (similar to Windows)
UseDllImport("__Internal")

Code initially based offNnanomsg:

staticLibraryLoader(){// Figure out which OS we're on. Windows or "other".if(Environment.OSVersion.Platform==PlatformID.Unix||Environment.OSVersion.Platform==PlatformID.MacOSX||// Legacy mono value. See https://www.mono-project.com/docs/faq/technical/(int)Environment.OSVersion.Platform==128){LoadPosixLibrary();}else{LoadWindowsLibrary();}}staticvoidLoadPosixLibrary(){constintRTLD_NOW=2;stringrootDirectory=AppDomain.CurrentDomain.BaseDirectory;stringassemblyDirectory=Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);// Environment.OSVersion.Platform returns "Unix" for Unix or OSX, so use RuntimeInformation herevarisOsx=System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.OSX);stringlibFile=isOsx?"libnng.dylib":"libnng.so";// x86 variants aren't in https://docs.microsoft.com/en-us/dotnet/core/rid-catalogstringarch=(isOsx?"osx":"linux")+"-"+(Environment.Is64BitProcess?"x64":"x86");// Search a few different locations for our native assemblyvarpaths=new[]{// This is where native libraries in our nupkg should end upPath.Combine(rootDirectory,"runtimes",arch,"native",libFile),// The build output folderPath.Combine(rootDirectory,libFile),Path.Combine("/usr/local/lib",libFile),Path.Combine("/usr/lib",libFile)};foreach(varpathinpaths){if(path==null){continue;}if(File.Exists(path)){varaddr=dlopen(path,RTLD_NOW);if(addr==IntPtr.Zero){// Not using NanosmgException because it depends on nn_errno.varerror=Marshal.PtrToStringAnsi(dlerror());thrownewException("dlopen failed: "+path+" : "+error);}NativeLibraryPath=path;return;}}thrownewException("dlopen failed: unable to locate library "+libFile+". Searched: "+paths.Aggregate((a,b)=>a+"; "+b));}[DllImport("libdl")]staticexternIntPtrdlopen(StringfileName,intflags);[DllImport("libdl")]staticexternIntPtrdlerror();[DllImport("libdl")]staticexternIntPtrdlsym(IntPtrhandle,Stringsymbol);

The use ofSystem.Runtime.InteropServices.RuntimeInformation came fromthis blog.

We construct a path based on the host OS (Linux vs OSX, 32 vs 64-bit), and pass it todlopen() to pre-load the shared library.

Note the absence of file extension with[DllImport("libdl")]. This will loadlibdl.dylib on OSX andlibdl.so on Linux.

Change the imported function to:

[DllImport("__Internal",EntryPoint="nng_rep0_open",CallingConvention=Cdecl)][return:MarshalAs(I4)]privatestaticexternint__Open(refuintsid);

Debug output:

Native library: /Users/jake/test/bin/Debug/runtimes/osx-x64/native/libnng.dylibMono: DllImport attempting to load: '__Internal'.Mono: DllImport loaded library '(null)'.Mono: DllImport searching in: '__Internal' ('(null)').Mono: Searching for 'nng_rep0_open'.Mono: Probing 'nng_rep0_open'.Mono: Found as 'nng_rep0_open'.

Works, but requiring everyDllImport use"__Internal" leaves a lot to be desired. There’s a few alternatives:

UseT4 template (or other script) to juggle the pinvoke imports needed by each platform
SettingDYLD_xxx_PATH
.config file with<dllmap>
Just copying the dylib to the output path as part of the build (arbitrary scripts can be included as part of a nupkg)

Again, this approach works, but has the following drawbacks:

Over-reliance on build/package tooling that can be a hassle to debug and get working correctly
On Windows, requires settingPlatform target to something other thanAny CPU
Argument toDllImport must be compile-time constant and requires massaging to get “magic” working on all platforms

One Load Context to Rule Them All

Came across“Best Practices for Assembly Loading”.

Started looking for information on these “load contexts” and foundan interesting document:

Custom LoadContext can override the AssemblyLoadContext.LoadUnmanagedDll method to intercept PInvokes from within the LoadContext instance so that can be resolved from custom binaries

Ho ho.

Also came acrossthis post of someone that works on ASP.NET Core.He’s usingAssemblyLoadContext to wrangle plugins, but mentionsLoadUnmanagedDll is“the only good way to load unmanaged binaries dynamically”.

To get started, needSystem.Runtime.Loader package:dotnet add nng.NETCore package system.runtime.loader

First attempt hard-coding paths and filenames:

publicclassALC:System.Runtime.Loader.AssemblyLoadContext{protectedoverrideAssemblyLoad(AssemblyNameassemblyName){if(assemblyName.Name=="nng.NETCore")returnLoadFromAssemblyPath("/Users/jake/nng.NETCore/bin/Debug/netstandard2.0/nng.NETCore.dll");// Return null to fallback on default load contextreturnnull;}protectedoverrideIntPtrLoadUnmanagedDll(stringunmanagedDllName){// Native nng shared libraryreturnLoadUnmanagedDllFromPath("/Users/jake/nng/build/libnng.dylib");}}

DllImported methods must bestatic so can’t useActivator.CreateInstance() to easily get at them. Could probably usereflection to extract them all, but that would be unwieldy.

I think the key is fromthat LoadContext design doc:

If an assembly A1 triggers the load of an assembly C1, the latter’s load is attempted within the LoadContext instance of the former

Basically, once a load context loads an assembly, subsequent dependent loads go through the same context. So, I moved a factory I use to create objects for my tests into the assembly with the pinvoke methods:

TestFactoryfactory;[Fact]publicasyncTaskPushPull(){varalc=newALC();varassem=alc.LoadFromAssemblyName(newSystem.Reflection.AssemblyName("nng.NETCore"));vartype=assem.GetType("nng.Tests.TestFactory");factory=(TestFactory)Activator.CreateInstance(type);//...

We can’t easily call thestatic pinvoke methods directly, but we can use a custom load context to instantiate a type which then calls the pinvokes.

I rarely find exceptions exciting, but this one is:

Exception thrown: 'System.InvalidCastException' in tests.dll: '[A]nng.Tests.TestFactory cannot be cast to [B]nng.Tests.TestFactory. Type A originates from 'nng.NETCore, Version=0.0.1.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' at location '/Users/jake/nng.NETCore/tests/bin/Debug/netcoreapp2.1/nng.NETCore.dll'. Type B originates from 'nng.NETCore, Version=0.0.1.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' at location '/Users/jake/nng.NETCore/tests/bin/Debug/netcoreapp2.1/nng.NETCore.dll'.'

Different load contexts, different types.

I’m referencing thenng.NETCore assembly (which contains the pinvokes) in my test project and also trying to load it here. How am I supposed to use a type I don’t know about? This is an opportunity for a C# feature I never use,dynamic:

dynamicfactory=Activator.CreateInstance(type);//...varpushSocket=factory.CreatePusher(url,true);

Test passes, hit breakpoints most of the places I expect (neither VS Code nor VS for Mac can hit breakpoints throughdynamic), but if I setDYLD_PRINT_LIBRARIES my assemblies are conspiculously absent:

dyld: loaded: /usr/local/share/dotnet/shared/Microsoft.NETCore.App/2.1.2/System.Globalization.Native.dylibdyld: loaded: /usr/local/share/dotnet/shared/Microsoft.NETCore.App/2.1.2/System.Native.dylibMicrosoft (R) Test Execution Command Line Tool Version 15.7.0Copyright (c) Microsoft Corporation. All rights reserved.dyld: loaded: /usr/local/share/dotnet/shared/Microsoft.NETCore.App/2.1.2/System.Security.Cryptography.Native.Apple.dylibStarting test execution, please wait...Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.Test Run Successful.Test execution time: 1.3259 Secondsdyld: unloaded: /usr/local/share/dotnet/shared/Microsoft.NETCore.App/2.1.2/libhostpolicy.dylibdyld: unloaded: /usr/local/share/dotnet/shared/Microsoft.NETCore.App/2.1.2/libhostpolicy.dylib

It would seemAssemblyLoadContext.LoadFrom*() doesn’t usedyld? Hmm… not sure about that.

Obviously, I don’t want to usedynamic all over the place. I refactored things; remove the test assembly reference to the pinvoke assembly, and introduce a “middle-man”/glue assembly containing interfaces both use:

Assembly	Project References	Dynamically Loads	Notes
“tests”	interfaces	pinvoke	Unit/integration tests
“interfaces”			`interface`s of high-level types using P/Invoke
“pinvoke”	interfaces		P/Invoke methods and wrapper types that use them

That enables me to write the very sane:

[Fact]publicasyncTaskPushPull(){varalc=newALC();varassem=alc.LoadFromAssemblyName(newSystem.Reflection.AssemblyName("nng.NETCore"));vartype=assem.GetType("nng.Tests.TestFactory");IFactory<NngMessage>factory=(IFactory<NngMessage>)Activator.CreateInstance(type);varpushSocket=factory.CreatePusher("ipc://test",true);

And now I can load native binaries from anywhere I like.

Out of curiousity, I wondered if I could add a reference back to the pinvoke, and after the native library had been successfully called use it directly:

Native.Msg.UnsafeNativeMethods.nng_msg_alloc(outvarmsg,UIntPtr.Zero);

Nope:

nng.Tests.PushPullTests.PushPull [FAIL][xUnit.net 00:00:00.5527750] System.DllNotFoundException : Unable to load shared library 'nng' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libnng, 1): image not found

The “default” load context knows not about the native library- it’s only in my custom context.

I get the feeling there may be a simpler way to achieve what I want. Need to investigate this a bit more.

Performance

None of this is “zero-cost”.

The NNanomsg source code referencesthis blog which mentions theSuppressUnmanagedCodeSecurity attribute may improve performance significantly. The security implications aren’t immediately clear from the documentation, but it sounds like it may only pertain to operating system resources; it can’t be any less safe than calling the library from native code…

There’snumerous methods to manipulate messages and I’ll write them in pure C# to avoid doing pinvoke for trivial string operations.

Top comments(6)

Satinder Singh

Dotnet Developer

Location
Mumbai
Joined
Dec 22, 2020

• Dec 22 '20

Copy link

Hi,
I must say this is nice detailed article, well I just came to know about a broken link in this post which pointing to one of my article. Can you update it with working link
i.ecodepedia.info/dotnet-core-to-dete...
instead of
json-formatter.codepedia.info/dotn...

Thanks & Happy Coding

jeikabu

Location
Santa Clara
Education
M.S.
Joined
Aug 14, 2018

• Feb 2 '21

Copy link

Done

eri0

Joined
Apr 30, 2018

• Oct 27 '19

Copy link

This was a great post, thanks for writing it. Running native code, with cross platform builds on C# and .NET is something that I couldn't find much material on the internet.

jeikabu

Location
Santa Clara
Education
M.S.
Joined
Aug 14, 2018

• Oct 28 '19

Copy link

Glad you enjoyed it. Wish I had more time to investigate it further.

paulcc

Did not think I'd be coding in my age. :-)

Education
Prague, Czech Republic
Work
C/C++ coding
Joined
Jan 29, 2022

• Jan 29 '22• Edited onJan 29• Edited

Copy link

Hello, I found your article while searching for solutions to the DllImport style difference in calling native code ( C/C++ ) between Windows and iOS.
I am trying to get common .netstandard2.0 implementation of native interface that needs to work both on Windows and iOS.
From reading your text, I got the impression that it's possible on Windows to preload a dll, which somehow makes the symbols from the DLL available, and then use the P/Invoke DllImport("__Internal") style. My thinking is that on the Windows I'd preload the DLL, and then proceed to using the same DllImport("__Internal") style, thus getting a single C# implementation working on both targets.
In my tests on Windows, the DllImport("__Internal") throws an exception complaining that the DLL does not exist - so it seems the "__Internal" is taken as DLL file name.

Could you, please, tell me if I understood your text correctly, and if so, suggest the simplest test code.
I am a C++ programmer just trying to figure this out....
Thanks, Paul.

Jim Borden

I currently maintain and develop Couchbase Lite as a remote worker in Tokyo.

Location
Tokyo
Education
B.S. Digital Media / Minor CS and Japanese
Work
Senior Software Engineer at Couchbase
Joined
Apr 17, 2018

• Oct 7 '19

Copy link

Glad somebody else is worry about this and not just me. In the end the "copy the files with nupkg script and then load based on IntPtr size" has served me pretty well!