Movatterモバイル変換

Beyond PortingHow Modern OpenGL canRadically Reduce Driver Overhead

Who are we?Cass Everitt, NVIDIA CorporationJohn McDonald, NVIDIA Corporation

What will we cover?Dynamic Buffer GenerationEfficient Texture ManagementIncreasing Draw Call Count

Dynamic Buffer GenerationProblemOur goal is to generate dynamic geometry directly in place.It will be used one time, and will be completely regenerated next frame.Particle systems are the most common exampleVegetation / foliage also common

Typical Solutionvoid UpdateParticleData(uint _dstBuf) {BindBuffer(ARRAY_BUFFER, _dstBuf);access = MAP_UNSYNCHRONIZED | MAP_WRITE_BIT;for particle in allParticles {dataSize = GetParticleSize(particle);void* dst = MapBuffer(ARRAY_BUFFER, offset, dataSize, access);(*(Particle*)dst) = *particle;UnmapBuffer(ARRAY_BUFFER);offset += dataSize;}};// Now render with everything.

The horrorvoid UpdateParticleData(uint _dstBuf) {BindBuffer(ARRAY_BUFFER, _dstBuf);access = MAP_UNSYNCHRONIZED | MAP_WRITE_BIT;for particle in allParticles {dataSize = GetParticleSize(particle);void* dst = MapBuffer(ARRAY_BUFFER, offset, dataSize, access);(*(Particle*)dst) = *particle;UnmapBuffer(ARRAY_BUFFER);This is so slow.offset += dataSize;}};// Now render with everything.

Driver interludeFirst, a quick interlude on modern GL driversIn the application (client) thread, the driver is very thin.It simply packages work to hand off to the server thread.The server thread does the real processingIt turns command sequences into push buffer fragments.

Healthy Driver Interaction VisualizedApplicationDriver (Client)Driver (Server)GPUThread separatorComponent separatorState ChangeAction Method (draw, clear, etc)Present

MAP_UNSYNCHRONIZEDAvoids an application-GPU sync point (a CPU-GPU sync point)But causes the Client and Server threads to serializeThis forces all pending work in the server thread to completeIt’s quite expensive (almost always needs to be avoided)

Client-Server Stall of SadnessApplicationDriver (Client)Driver (Server)GPUThread separatorComponent separatorState ChangeAction Method (draw, clear, etc)Present

It’s okayQ: What’s better than mapping in an unsynchronized manner?A: Keeping around a pointer to GPU-visible memory forever.Introducing: ARB_buffer_storage

ARB_buffer_storageConceptually similar to ARB_texture_storage (but for buffers)Creates an immutable pointer to storage for a bufferThe pointer is immutable, the contents are not.So BufferData cannot be called—BufferSubData is still okay.Allows for extra information at create time.For our usage, we care about the PERSISTENT and COHERENTbits.PERSISTENT: Allow this buffer to be mapped while the GPU is using it.COHERENT: Client writes to this buffer should be immediately visible tothe GPU.http://www.opengl.org/registry/specs/ARB/buffer_storage.txt

ARB_buffer_storage cont’dAlso affects the mapping behavior (pass persistent and coherentbits to MapBufferRange)Persistently mapped buffers are good for:Dynamic VB / IB dataHighly dynamic (~per draw call) uniform dataMulti_draw_indirect command buffers (more on this later)Not a good fit for:Static geometry buffersLong lived uniform data (still should use BufferData or BufferSubData forthis)

Armed with persistently mapped buffers// At the beginning of timeflags = MAP_WRITE_BIT | MAP_PERSISTENT_BIT | MAP_COHERENT_BIT;BufferStorage(ARRAY_BUFFER, allParticleSize, NULL, flags);mParticleDst = MapBufferRange(ARRAY_BUFFER, 0, allParticleSize,flags);mOffset = 0;// allParticleSize should be ~3x one frame’s worth of particles// to avoid stalling.

Update Loop (old and busted)void UpdateParticleData(uint _dstBuf) {BindBuffer(ARRAY_BUFFER, _dstBuf);access = MAP_UNSYNCHRONIZED | MAP_WRITE_BIT;for particle in allParticles {dataSize = GetParticleSize(particle);void* dst = MapBuffer(ARRAY_BUFFER, offset, dataSize, access);(*(Particle*)dst) = *particle;offset += dataSize;UnmapBuffer(ARRAY_BUFFER);}};// Now render with everything.

Update Loop (new hotness)void UpdateParticleData() {for particle in allParticles {dataSize = GetParticleSize(particle);mParticleDst[mOffset] = *particle;mOffset += dataSize; // Wrapping not shown}};// Now render with everything.

Performance results160,000 point spritesSpecified in groups of 6 vertices (one particle at a time)Synthetic (naturally)MethodFPSParticles / SMap(UNSYNCHRONIZED)1.369219,040BufferSubData17.652,824,000D3D11 Map(NO_OVERWRITE)20.253,240,000

Performance results160,000 point spritesSpecified in groups of 6 vertices (one particle at a time)Synthetic (naturally)MethodFPSParticles / SMap(UNSYNCHRONIZED)1.369219,040BufferSubData17.652,824,000D3D11 Map(NO_OVERWRITE)20.253,240,000Map(COHERENT|PERSISTENT)79.912,784,000Room for improvement still, but much, much better.

The other shoeYou are responsible for not stomping on data in flight.Why 3x?1x: What the GPU is using right now.2x: What the driver is holding, getting ready for the GPU to use.3x: What you are writing to.3x should ~ guarantee enough buffer room*…Use fences to ensure that rendering is complete before you beginto write new data.

FencingUse FenceSync to place a new fence.When ready to scribble over that memory again, useClientWaitSync to ensure that memory is done.ClientWaitSync will block the client thread until it is readySo you should wrap this function with a performance counterAnd complain to your log file (or resize the underlying buffer) if youfrequently see stalls hereFor complete details on correct management of buffers withfencing, see Efficient Buffer Management [McDonald 2012]

Efficient Texture ManagementOr ―how to manage all texture memory myself‖

ProblemChanging textures breaks batches.Not all texture data is needed all the timeTexture data is large (typically the largest memory bucket for games)Bindless solves this, but can hurt GPU performanceToo many different textures can fall out of TexHdr$Not a bindless problem per se

TerminologyReserve – The act of allocating virtual memoryCommit – Tying a virtual memory allocation to a physical backingstore (Physical memory)Texture Shape – The characteristics of a texture that affect itsmemory consumptionSpecifically: Height, Width, Depth, Surface Format, Mipmap Level Count

Old SolutionTexture AtlasesProblemsCan impact art pipelineTexture wrap, border filteringColor bleeding in mip maps

Texture ArraysIntroduced in GL 3.0, and D3D 10.Arrays of textures that are the same shape and formatTypically can contain many ―layers‖ (2048+)Filtering works as expectedAs does mipmapping!

Sparse Bindless Texture ArraysOrganize loose textures into Texture Arrays.Sparsely allocate Texture ArraysIntroducing ARB_sparse_textureConsume virtual memory, but not physical memoryUse Bindless handles to deal with as many arrays as needed!Introducing ARB_bindless_textureuncommitteduncommitteduncommittedlayerlayerlayer

ARB_sparse_textureApplications get fine-grained control of physicalmemory for textures with large virtual allocationsInspired by Mega TexturePrimary expected use cases:Sparse texture dataTexture pagingDelayed-loading assetshttp://www.opengl.org/registry/specs/ARB/sparse_texture.txt

ARB_bindless_textureTextures specified by GPU-visible ―handle‖ (really an address)Rather than by name and binding pointCan come from ~anywhereUniformsVaryingSSBOOther texturesTexture residency also application-controlledResidency is ―does this live on the GPU or in sysmem?‖https://www.opengl.org/registry/specs/ARB/bindless_texture.txt

AdvantagesArtists work naturallyNo preprocessing required (no bake-step required)Although preprocessing is helpful if ARB_sparse_texture is unavailableReduce or eliminate TexHdr$ thrashingEven as compared to traditional texturingProgrammers manage texture residencyWorks well with arbitrary streamingFaster on the CPUFaster on the GPU

DisadvantagesTexture addresses are now structs (96 bits).64 bits for bindless handle32 bits for slice index (could reduce this to 10 bits at a perf cost)ARB_sparse_texture implementations are a bit immatureEarly adopters: please bring us your bugs.ARB_sparse_texture requires base level be a multiple of tile size(Smaller is okay)Tile size is queried at runtimeTextures that are power-of-2 should almost always be safe.

Implementation OverviewWhen creating a new texture…Check to see if any suitable texture array existsTexture arrays can contain a large number of textures of the same shapeEx. Many TEXTURE_2Ds grouped into a single TEXTURE_2D_ARRAYIf no suitable texture, create a new one.

Texture Container Creation (example)GetIntegerv( MAX_SPARSE_ARRAY_TEXTURE_LAYERS, maxLayers );Choose a reasonable size (e.g. array size ~100MB virtual )If new internalFormat, choose page sizeGetInternalformativ( …, internalformat, NUM_VIRTUAL_PAGE_SIZES, 1, &numIndexes);Note: numIndexes can be 0, so have a planIterate, select suitable pageSizeIndexBindTexture( TEXTURE_2D_ARRAY, newTexArray );TexParameteri( TEXTURE_SPARSE, TRUE );TexParameteri( VIRTUAL_PAGE_SIZE_INDEX, pageSizeIndex );Allocate the texture’s virtual memory using TexStorage3D

Specifying Texture DataUsing the located/created texture array from the previous stepAllocate a layer as the location of our dataFor each mipmap level of the allocated layer:Commit the entire mipmap level (using TexPageCommitment)Specify actual texel data as usual for arraysgl(Compressed|Copy|)TexSubImage3DPBO updates are fine toouncommittedAllocated layerfreefreelayersliceslice

Freeing TexturesTo free the texture, reverse the process:Use TexPageCommitment to mark the entire layer (slice) as free.Do once for each mipmap levelAdd the layer to the free list for future allocationuncommittedfreefreelayerslicesliceuncommittedFreed layerlayer

Combining with Bindless to eliminate bindsAt container create time:Specify sampling parameters via SamplerParameter calls firstCall GetTextureSamplerHandleARB to return a GPU-visible pointer to thetexture+sampler containerCall MakeTextureHandleResident to ensure the resource lives on the GPUAt delete time, call MakeTextureHandleNonResidentWith bindless, you explicitly manage the GPU’s working set

Using texture data in shadersWhen a texture is needed with the default sampling parametersCreate a GLSL-visible TextureRef object:struct TextureRef {sampler2DArray container;float slice;};When a texture is needed with custom sampling parametersCreate a separate sampler object for the shader with the parametersCreate a bindless handle to the pair usingGetTextureSamplerHandle, then call MakeTextureHandleResidentwith the new valueAnd fill out a TextureRef as above for usage by GLSL

C++ CodeBasic implementation (some details missing)BSD licensed (use as you will)https://github.com/nvMcJohn/apitest/blob/pdoane_newtests/sparse_bindless_texarray.hhttps://github.com/nvMcJohn/apitest/blob/pdoane_newtests/sparse_bindless_texarray.cpp

Increasing Draw Call CountLet’s draw all the calls!

All the Draw Calls!ProblemYou want more draw calls of smaller objects.D3D is slow at this.Naïve GL is faster than D3D, but not fast enough.

XY ProblemY: How can I have more draw calls?X: You don’t really care if it’s more draw calls, right?Really what you want is to be able to draw more small geometrygroupings. More objects.

Well why didn’t you just say so??First, some background.What makes draw calls slow?Real world API usageDraw Call Cost Visualization

Some backgroundWhat causes slow draw calls?Validation is the biggest bucket (by far).Pre-validation is ―difficult‖―Every application does the same things.‖Not really. Most applications are in completely disjoint statesTry this experiment: What is important to you?Now ask your neighbor what’s important to him.

Why is prevalidation difficult?The GPU is an exceedingly complex state machine.(Honestly, it’s probably the most complex state machine in all of CS)Any one of those states may have a problem that requires WARUsually the only problem is overall performanceBut sometimes not. There are millions of tests covering NVIDIA GPU functionality.

FINE.How can app devs mitigate these costs?Minimize state changes.All state changes are not created equal!Cost of a draw call:Small fixed cost + Cost of validation of changed state

Feels limiting…Artists want lots of materials, and small amounts of geometryEven better: What if artists just didn’t have to care about this?Ideal Programmer->Artist Interaction―You make pretty art. I’ll make it fit.‖

Relative costs of State ChangesIn decreasing cost…Render TargetProgramROPTexture BindingsVertex FormatUBO BindingsVertex BindingsUniform Updates~60K / s~300K / s~1.5M / s~10M / sNote: Not to scale

Real World API frequencyAPI usage looks roughly like this…Increasing Frequency of ChangeRender Target (scene)Per Scene Uniform Buffer + TexturesIB / VB and Input LayoutShader (Material)Per-material Uniform Buffer + TexturesPer-object Uniform Buffer + TexturesPer-piece Uniform Buffer + TexturesDraw

Draw Calls visualizedRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

Draw Calls visualized (cont’d)Read down, then rightBlack—no changeRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

GoalsLet’s minimize validation costs without affecting artistsThings we need to be fast (per app call frequency):Uniform Updates and bindingTexture Updates and bindingThese happen most often in app, ergo driving them to ~0 shouldbe a win.

TexturesUsing Sparse Bindless Texture Arrays (as previously described)solves this.All textures are set before any drawing begins(No need to change textures between draw calls)Note that from the CPU’s perspective, just using bindless issufficient.That was easy.

Eliminating Texture Binds -- visualizedIncreasing Frequency of ChangeRender Target (scene)Per Scene Uniform Buffer + TexturesIB / VB and Input LayoutShader (Material)Per-material Uniform Buffer + TexturesPer-object Uniform Buffer + TexturesPer-piece Uniform Buffer + TexturesDrawRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

Boom!Increasing Frequency of ChangeRender Target (scene)Per Scene Uniform BufferIB / VB and Input LayoutShader (Material)Per-material Uniform BufferPer-object Uniform BufferPer-piece Uniform BufferDrawRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

Buffer updates (old and busted)Typical Scene Graph Traversalfor obj in visibleObjectSet {update(buffer, obj);draw(obj);}

Buffer updates (new hotness)Typical Scene Graph Traversalfor obj in visibleObjectSet {update(bufferFragment, obj);}for obj in visibleObjectSet {draw(obj);}

bufferFragma-wha?Rather than one buffer per object, we share UBOs for manyobjects.ie, given struct ObjectUniforms { /* … */ };// Old (probably not explicitly instantiated,// just scattered in GLSL)ObjectUniforms uniformData;// NewObjectUniforms uniformData[ObjectsPerKickoff];Use persistent mapping for even more win here!For large amounts of data (bones) consider SSBO.Introducing ARB_shader_storage_buffer_object

SSBO?Like ―large‖ uniform buffer objects.Minimum required size to claim support is 16M.Accessed like uniforms in shaderSupport for better packing (std430)Caveat: They are typically implemented in hardware as textures(and can introduce dependent texture reads)Just one of a laundry list of things to consider, not to discourage use.http://www.opengl.org/registry/specs/ARB/shader_storage_buffer_object.txt

Eliminating Buffer Update OverheadIncreasing Frequency of ChangeRender Target (scene)Per Scene Uniform BufferIB / VB and Input LayoutShader (Material)Per-material Uniform BufferPer-object Uniform BufferPer-piece Uniform BufferDrawRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

Sweet!Increasing Frequency of ChangeRender Target (scene)IB / VB and Input LayoutShader (Material)Draw ( * each object )Hrrrrmmmmmm….Render TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

So now…It’d be awesome if we could do all of those kickoffs at once.Validation is already only paid onceBut we could just pay the constant startup cost once.If only…….

So now…It’d be awesome if we could do all of those kickoffs at once.Validation is already only paid onceBut we could just pay the constant startup cost once.If only…….Introducing ARB_multi_draw_indirect

ARB_multi_draw_indirectAllows you to specify parameters to draw commands from abuffer.This means you can generate those parameters wide (on the CPU)Or even on the GPU, via compute program.http://www.opengl.org/registry/specs/ARB/multi_draw_indirect.txt

ARB_multi_draw_indirect cont’dvoid MultiDrawElementsIndirect(enum mode,enum typeconst void* indirect,sizei primcount,sizei stride);

ARB_multi_draw_indirect cont’dconst ubyte * ptr = (const ubyte *)indirect;for (i = 0; i < primcount; i++) {DrawArraysIndirect(mode,(DrawArraysIndirectCommand*)ptr);if (stride == 0){ptr += sizeof(DrawArraysIndirectCommand);} else {ptr += stride;}}

DrawArraysIndirectCommandtypedef struct {uint count;uint primCount;uint first;uint baseInstance;} DrawArraysIndirectCommand;

Knowing which shader data is mineUse ARB_shader_draw_parameters, a necessary companion toARB_multi_draw_indirectAdds a builtin to the VS: DrawID (InstanceID already available)This tells you which command of a MultiDraw command is beingexecuted.When not using MultiDraw, the builtin is specified to be 0.Caveat: Right now, you have to pass this down to other shaderstages as an interpolant.Hoping to have that rectified via ARB or EXT extension ―real soon now.‖http://www.opengl.org/registry/specs/ARB/shader_draw_parameters.txt

Applying everythingCPU Perf is massively better5-30x increase in number of distinct objects / sInteraction with driver is decreased ~75%Note: GPU perf can be affected negatively (although not toobadly)As always: Profile, profile, profile.

Previous ResultsRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

Visualized ResultsRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex FormatMultiDraw

Where we came fromRender TargetTextureUniform UpdatesProgramUBO BindingDrawROPVertex Format

ConclusionGo forth and work magnify.

Questions?jmcdonald at nvidia dot comcass at nvidia dot com

Movatterモバイル変換

Change Language

Beyond porting

Embed presentation

Recommended

More Related Content

What's hot

Similar to Beyond porting

Recently uploaded

In this document

Beyond porting

Editor's Notes