mirror of
https://github.com/bkaradzic/bgfx.git
synced 2026-02-17 12:42:34 +01:00
Feature/d3d4linux integration (#3578)
* Add d3d4linux to 3rdparty for Linux HLSL shader compilation - Added d3d4linux wrapper library for D3DCompiler access on Linux - Includes Microsoft D3DCompiler DLLs (versions 43 and 47) - Source from https://github.com/samhocevar/d3d4linux - Commit: c9b1ca9 (Handle D3DReflect() regardless of DLL version) - Enables HLSL shader cross-compilation on Linux via Wine This is the first step toward implementing issue #1869: Enable DirectX shader compilation on Linux platforms without requiring Windows machines in the build pipeline. Components included: - d3d4linux.cpp - Wine-based D3DCompiler wrapper - d3dcompiler_43.dll & d3dcompiler_47.dll - MS compiler DLLs - Header files for D3D API compatibility - Makefile for building the wrapper library - Test files for validation * Fix d3d4linux struct alignment for D3DReflect compatibility - Add D3D_MIN_PRECISION enum to d3d4linux_enums.h - Add MinPrecision field and padding to D3D11_SIGNATURE_PARAMETER_DESC - Fixes struct size mismatch (32 bytes -> 40 bytes) with d3dcompiler_47.dll - All 4 D3D functions now work: D3DCompile, D3DReflect, D3DStripShader, D3DDisassemble - Add implementation_log.md documenting the integration process Part of issue #1869: Enable HLSL shader compilation on Linux via Wine * Integrate d3d4linux with bgfx shaderc for Linux HLSL compilation Add support for HLSL shader compilation on Linux via d3d4linux (Wine-based). This enables building D3D11/D3D12 shaders on Linux without Windows. Changes: - shaderc.h: Add SHADERC_CONFIG_HLSL_D3D4LINUX option to enable HLSL on Linux - shaderc_hlsl.cpp: Add d3d4linux code path with conditional compilation - Uses d3d4linux inline functions instead of Windows DLL loading - load() returns d3d4linux compiler info on Linux - unload() is no-op on d3d4linux (no DLL to close) - shaderc.lua: Add d3d4linux include path for Linux/macOS builds Usage: Build with -DSHADERC_CONFIG_HLSL_D3D4LINUX=1 Set D3D4LINUX_WINE, D3D4LINUX_EXE, D3D4LINUX_DLL environment variables Requires: Wine, d3d4linux.exe, d3dcompiler_47.dll Part of issue #1869: Enable HLSL shader compilation on Linux * d3d4linux: Complete shaderc HLSL integration with Wine Full working HLSL shader compilation on Linux via d3dcompiler_47.dll and Wine. d3d4linux fixes: - Add D3D_MIN_PRECISION enum and MinPrecision field to D3D11_SIGNATURE_PARAMETER_DESC (fixes struct size mismatch: 32 bytes -> 40 bytes to match Windows ABI) - Add missing D3DCOMPILE_* flags used by shaderc - Add ID3D11ShaderReflectionType struct with GetType() support - Wine 11+ compatibility: default path changed from wine64 to wine with fallback - Fix D3DDisassemble IPC protocol bug (only write comment string when present) shaderc integration: - Add SHADERC_CONFIG_HLSL_D3D4LINUX=1 define for Linux/macOS builds - Restructure shaderc_hlsl.cpp with conditional compilation for d3d4linux vs Windows - Fix null pointer crash when D3DCompile returns error without message - Initialize ID3DBlob pointers to NULL Usage: D3D4LINUX_EXE=/path/to/d3d4linux.exe shaderc -f shader.sc -o out.bin \ --type vertex --platform windows -p s_5_0 Tested with vs_cubes.sc, fs_cubes.sc, vs_bump.sc - all compile successfully. D3DCompile, D3DReflect, D3DStripShader, D3DDisassemble all working. Implements: bkaradzic/bgfx#1869 * docs: Add d3d4linux documentation - docs/d3d4linux-shaderc-support.md: User guide explaining prerequisites, installation, usage, environment variables, and troubleshooting for compiling HLSL shaders on Linux/macOS via Wine - implementation_log.md: Updated with shaderc integration completion details, including all code changes made, test results, and commit references * docs: Add EditorConfig/eclint notes to implementation log Documents that eclint reports some style issues in modified files, but these are pre-existing in the original codebase (copyright header comments using spaces, line length in shaderc files). Our changes follow the existing code style and do not introduce new violations. * Fix d3d4linux integration after rebase onto upstream master This commit fixes build issues that arose after rebasing onto the updated upstream master which includes the TINT library and directx-headers. Changes in this commit: ## shaderc.h - Disable DXIL (DXC compiler) when d3d4linux is enabled - DXIL uses DirectX Compiler (DXC) which requires directx-headers - d3d4linux uses D3DCompiler via Wine which is a different compilation path - These two approaches are mutually exclusive on Linux ## shaderc.lua - Removed directx-headers include paths for Linux builds - d3d4linux provides its own D3D type definitions - directx-headers conflicts with d3d4linux headers (duplicate typedefs) ## shaderc_hlsl.cpp - Made all D3D-related includes conditional on SHADERC_CONFIG_HLSL_D3D4LINUX - Use <d3d4linux.h> when d3d4linux is enabled - Use <d3dcompiler.h> and <d3d11shader.h> on Windows - PFN_D3D_* function pointer typedefs (use WINAPI calling convention) - D3DCompiler struct with IID_ID3D11ShaderReflection member - s_d3dcompiler[] array of DLL versions - s_d3dcompilerdll handle - Added simplified D3DCompiler struct for d3d4linux path (fileName only) - Fixed D3DReflect call to use d3d4linux's integer IID macro ## Testing - Successfully compiled shaderc with SHADERC_CONFIG_HLSL_D3D4LINUX=1 - Successfully compiled vs_cubes.sc to D3D11 bytecode (DXBC format) - Output verified with xxd showing VSH header and DXBC signature * Cleanup d3d4linux integration per maintainer feedback Simplified changes: - Remove SHADERC_CONFIG_HLSL_D3D4LINUX define, use BX_PLATFORM_* directly - SHADERC_CONFIG_HLSL now enabled on Windows/Linux/macOS - SHADERC_CONFIG_DXIL now Windows-only (DXC needs native D3D) Removed files: - d3d4linux test files (compile-hlsl.cpp, ps_sample.hlsl) - d3d4linux extras (UE4 patches) - d3d4linux.Build.cs (UE4 build file) - d3dcompiler_43.dll (only 47 needed) - implementation_log.md - docs/d3d4linux-shaderc-support.md Build and test verified on Linux with Wine 11.0. * Enable DXIL on Linux/macOS and configure d3d4linux paths Changes: - shaderc.h: Enable SHADERC_CONFIG_DXIL on Linux and macOS (previously Windows-only) This allows Shader Model 6.0+ compilation via DXC on non-Windows platforms - shaderc.lua: Configure d3d4linux and directx-headers for Linux/macOS - Add directx-headers include paths for DXIL support - Add WSL stubs include path for COM compatibility types - Define D3D4LINUX_EXE path pointing to 3rdparty/d3d4linux/d3d4linux.exe - Define D3D4LINUX_DLL path for d3dcompiler_47.dll (Wine Z: drive prefix) Prerequisites for HLSL compilation on Linux: 1. Wine installed (/usr/bin/wine or /usr/bin/wine64) 2. Build d3d4linux.exe: cd 3rdparty/d3d4linux && make (requires mingw-w64 cross-compiler: x86_64-w64-mingw32-c++) 3. d3dcompiler_47.dll in 3rdparty/d3d4linux/ Tested: SM 5.0 vertex shader compilation works via Wine/d3d4linux DXIL (SM 6.0+) requires native DXC library installation * Remove macOS from DXIL config (no DXC library available) Microsoft's DXC releases only include: - Windows: dxcompiler.dll - Linux: libdxcompiler.so There is no libdxcompiler.dylib for macOS. Updated SHADERC_CONFIG_DXIL to only enable DXIL on Windows and Linux. macOS still supports legacy HLSL (SM 5.0) via d3d4linux. * Remove d3d4linux build files from bgfx repo Users should obtain d3d4linux.exe and d3dcompiler_47.dll from: https://github.com/killerdevildog/d3d4linux Only headers are needed for bgfx compilation. * Fix d3d4linux reflection: deserialize D3D11_SHADER_TYPE_DESC Fixes uniform extraction bug where type info was not being read from the IPC stream. This is required for bgfx to properly determine uniform types and array element counts via findUniformType(). * Add d3d4linux source with reflection serialization fix Include d3d4linux.cpp and Makefile with fixes for D3D11_SHADER_TYPE_DESC serialization. This allows building shaderc with proper uniform extraction while upstream PR is pending: https://github.com/samhocevar/d3d4linux/pull/1 * Apply maintainer cleanup: unified code paths and header guards * Force D3D_SVF_USED flag on all uniform variables This forces all uniform variables to be marked as 'used' during reflection deserialization, ensuring bgfx extracts type info for all uniforms. Note: This deviates from native D3D compiler behavior where unused uniforms do not have the D3D_SVF_USED flag set. Native behavior would skip extraction for uniforms not actually referenced in shader code. * Fix D3DDisassemble serialization mismatch Only write comment string when szComments is non-null to match server-side read behavior. Fixes 'bad message received' error. * shaderc: Improve d3d4linux integration and error handling This commit enhances the d3d4linux integration for HLSL shader compilation on Linux and macOS platforms with several key improvements: Key Changes: - Dynamic path detection for d3d4linux binaries instead of hardcoded paths - Relocate d3dcompiler_47.dll from 3rdparty/d3d4linux/ to tools/bin/windows/ - Remove d3d4linux.exe from 3rdparty (now managed separately) - Improved error handling with proper null checks and user feedback - Enhanced logging using BX_TRACE for consistency - Remove --verbose flag from shader compilation make rules for cleaner output - Automatic environment variable setup (D3D4LINUX_EXE, D3D4LINUX_DLL) Technical Details: - shaderc_hlsl.cpp now dynamically locates d3d4linux.exe and d3dcompiler_47.dll relative to the executable directory - Proper error messages when required files are not found - Simplified Makefile in 3rdparty/d3d4linux/ - Removed hardcoded D3D4LINUX_EXE and D3D4LINUX_DLL defines from shaderc.lua Files Modified: - 3rdparty/d3d4linux/Makefile: Simplified build rules - scripts/shader.mk: Removed --verbose flags from compilation commands - scripts/shaderc.lua: Removed hardcoded path defines - tools/shaderc/shaderc_hlsl.cpp: Enhanced d3d4linux integration logic Files Deleted: - 3rdparty/d3d4linux/d3d4linux.exe - 3rdparty/d3d4linux/d3dcompiler_47.dll This improves portability and makes the build system more maintainable by removing hardcoded paths and improving runtime binary discovery. Credit: bkaradzic provided patch * Delete d3d4linux.md * Delete examples/runtime/shaders/dx11/vs_cubes.bin * Remove macOS. --------- Co-authored-by: Branimir Karadžić <branimirkaradzic@gmail.com>
This commit is contained in:
Quaylyn Rimer
@@ -11,14 +11,24 @@ namespace bgfx
|
||||
extern bool g_verbose;
|
||||
}
|
||||
|
||||
// HLSL compilation support:
|
||||
// - Windows: Native D3DCompiler DLL
|
||||
// - Linux/macOS: d3d4linux (Wine-based D3DCompiler via IPC)
|
||||
#ifndef SHADERC_CONFIG_HLSL
|
||||
# define SHADERC_CONFIG_HLSL BX_PLATFORM_WINDOWS
|
||||
# define SHADERC_CONFIG_HLSL (0 \
|
||||
|| BX_PLATFORM_WINDOWS \
|
||||
|| BX_PLATFORM_LINUX \
|
||||
)
|
||||
#endif // SHADERC_CONFIG_HLSL
|
||||
|
||||
// DXIL compilation support (Shader Model 6.0+):
|
||||
// - Windows: Native DXC (dxcompiler.dll)
|
||||
// - Linux: DXC (libdxcompiler.so) via directx-headers
|
||||
// - macOS: Not supported (no DXC dynamic library available)
|
||||
#ifndef SHADERC_CONFIG_DXIL
|
||||
# define SHADERC_CONFIG_DXIL (0 \
|
||||
|| BX_PLATFORM_WINDOWS \
|
||||
|| BX_PLATFORM_LINUX \
|
||||
# define SHADERC_CONFIG_DXIL (0 \
|
||||
|| BX_PLATFORM_WINDOWS \
|
||||
|| BX_PLATFORM_LINUX \
|
||||
)
|
||||
#endif // SHADERC_CONFIG_DXIL
|
||||
|
||||
|
||||
@@ -14,8 +14,13 @@
|
||||
#endif // defined(__MINGW32__)
|
||||
|
||||
#define COM_NO_WINDOWS_H
|
||||
#include <d3dcompiler.h>
|
||||
#include <d3d11shader.h>
|
||||
#if BX_PLATFORM_LINUX || BX_PLATFORM_OSX
|
||||
# include <wsl/winadapter.h>
|
||||
# include <d3d4linux.h>
|
||||
#else
|
||||
# include <d3dcompiler.h>
|
||||
# include <d3d11shader.h>
|
||||
#endif // BX_PLATFORM_LINUX || BX_PLATFORM_OSX
|
||||
#include <bx/os.h>
|
||||
|
||||
#ifndef D3D_SVF_USED
|
||||
@@ -24,36 +29,40 @@
|
||||
|
||||
namespace bgfx { namespace hlsl
|
||||
{
|
||||
typedef HRESULT(WINAPI* PFN_D3D_COMPILE)(_In_reads_bytes_(SrcDataSize) LPCVOID pSrcData
|
||||
, _In_ SIZE_T SrcDataSize
|
||||
, _In_opt_ LPCSTR pSourceName
|
||||
, _In_reads_opt_(_Inexpressible_(pDefines->Name != NULL) ) CONST D3D_SHADER_MACRO* pDefines
|
||||
, _In_opt_ ID3DInclude* pInclude
|
||||
, _In_opt_ LPCSTR pEntrypoint
|
||||
, _In_ LPCSTR pTarget
|
||||
, _In_ UINT Flags1
|
||||
, _In_ UINT Flags2
|
||||
, _Out_ ID3DBlob** ppCode
|
||||
, _Always_(_Outptr_opt_result_maybenull_) ID3DBlob** ppErrorMsgs
|
||||
typedef HRESULT(WINAPI* PFN_D3D_COMPILE)(
|
||||
LPCVOID pSrcData
|
||||
, SIZE_T SrcDataSize
|
||||
, LPCSTR pSourceName
|
||||
, CONST D3D_SHADER_MACRO* pDefines
|
||||
, ID3DInclude* pInclude
|
||||
, LPCSTR pEntrypoint
|
||||
, LPCSTR pTarget
|
||||
, UINT Flags1
|
||||
, UINT Flags2
|
||||
, ID3DBlob** ppCode
|
||||
, ID3DBlob** ppErrorMsgs
|
||||
);
|
||||
|
||||
typedef HRESULT(WINAPI* PFN_D3D_DISASSEMBLE)(_In_reads_bytes_(SrcDataSize) LPCVOID pSrcData
|
||||
, _In_ SIZE_T SrcDataSize
|
||||
, _In_ UINT Flags
|
||||
, _In_opt_ LPCSTR szComments
|
||||
, _Out_ ID3DBlob** ppDisassembly
|
||||
typedef HRESULT(WINAPI* PFN_D3D_DISASSEMBLE)(
|
||||
LPCVOID pSrcData
|
||||
, SIZE_T SrcDataSize
|
||||
, UINT Flags
|
||||
, LPCSTR szComments
|
||||
, ID3DBlob** ppDisassembly
|
||||
);
|
||||
|
||||
typedef HRESULT(WINAPI* PFN_D3D_REFLECT)(_In_reads_bytes_(SrcDataSize) LPCVOID pSrcData
|
||||
, _In_ SIZE_T SrcDataSize
|
||||
, _In_ REFIID pInterface
|
||||
, _Out_ void** ppReflector
|
||||
typedef HRESULT(WINAPI* PFN_D3D_REFLECT)(
|
||||
LPCVOID pSrcData
|
||||
, SIZE_T SrcDataSize
|
||||
, REFIID pInterface
|
||||
, void** ppReflector
|
||||
);
|
||||
|
||||
typedef HRESULT(WINAPI* PFN_D3D_STRIP_SHADER)(_In_reads_bytes_(BytecodeLength) LPCVOID pShaderBytecode
|
||||
, _In_ SIZE_T BytecodeLength
|
||||
, _In_ UINT uStripFlags
|
||||
, _Out_ ID3DBlob** ppStrippedBlob
|
||||
typedef HRESULT(WINAPI* PFN_D3D_STRIP_SHADER)(
|
||||
LPCVOID pShaderBytecode
|
||||
, SIZE_T BytecodeLength
|
||||
, UINT uStripFlags
|
||||
, ID3DBlob** ppStrippedBlob
|
||||
);
|
||||
|
||||
PFN_D3D_COMPILE D3DCompile;
|
||||
@@ -64,7 +73,7 @@ namespace bgfx { namespace hlsl
|
||||
struct D3DCompiler
|
||||
{
|
||||
const char* fileName;
|
||||
const GUID IID_ID3D11ShaderReflection;
|
||||
const GUID reflectionGuid;
|
||||
};
|
||||
|
||||
static const D3DCompiler s_d3dcompiler[] =
|
||||
@@ -78,10 +87,59 @@ namespace bgfx { namespace hlsl
|
||||
};
|
||||
|
||||
static const D3DCompiler* s_compiler = NULL;
|
||||
|
||||
#if BX_PLATFORM_WINDOWS
|
||||
static void* s_d3dcompilerdll = NULL;
|
||||
#endif // BX_PLATFORM_WINDOWS
|
||||
|
||||
const D3DCompiler* load(bx::WriterI* _messageWriter)
|
||||
{
|
||||
#if BX_PLATFORM_LINUX || BX_PLATFORM_OSX
|
||||
BX_TRACE("Using d3d4linux for HLSL compilation (Wine-based D3DCompiler).");
|
||||
|
||||
bx::Error messageErr;
|
||||
bx::FileInfo fi;
|
||||
|
||||
bx::FilePath executableDir = bx::FilePath(bx::Dir::Executable).getPath();
|
||||
|
||||
bx::FilePath d3d4linuxExe(executableDir);
|
||||
d3d4linuxExe.join("../windows/d3d4linux.exe");
|
||||
|
||||
if (bx::stat(fi, d3d4linuxExe) )
|
||||
{
|
||||
bx::setEnv("D3D4LINUX_EXE", d3d4linuxExe);
|
||||
}
|
||||
else
|
||||
{
|
||||
bx::write(_messageWriter, &messageErr, "Error: `%s` not found.\n", d3d4linuxExe.getCPtr() );
|
||||
return NULL;
|
||||
}
|
||||
|
||||
bx::FilePath d3d4linuxDll(executableDir);
|
||||
d3d4linuxDll.join("../windows/d3dcompiler_47.dll");
|
||||
|
||||
if (bx::stat(fi, d3d4linuxDll) )
|
||||
{
|
||||
bx::setEnv("D3D4LINUX_DLL", d3d4linuxDll);
|
||||
}
|
||||
else
|
||||
{
|
||||
bx::write(_messageWriter, &messageErr, "Error: `%s` not found.\n", d3d4linuxDll.getCPtr() );
|
||||
return NULL;
|
||||
}
|
||||
|
||||
if (g_verbose)
|
||||
{
|
||||
bx::setEnv("D3D4LINUX_VERBOSE", "1");
|
||||
}
|
||||
|
||||
D3DCompile = (PFN_D3D_COMPILE )&d3d4linux::compile;
|
||||
D3DReflect = (PFN_D3D_REFLECT )&d3d4linux::reflect;
|
||||
D3DDisassemble = (PFN_D3D_DISASSEMBLE )&d3d4linux::disassemble;
|
||||
D3DStripShader = (PFN_D3D_STRIP_SHADER)&d3d4linux::strip_shader;
|
||||
|
||||
return &s_d3dcompiler[0];
|
||||
#else
|
||||
if (NULL != s_d3dcompilerdll)
|
||||
{
|
||||
return s_compiler;
|
||||
@@ -109,6 +167,7 @@ namespace bgfx { namespace hlsl
|
||||
|| NULL == D3DStripShader)
|
||||
{
|
||||
bx::dlclose(s_d3dcompilerdll);
|
||||
s_d3dcompilerdll = NULL;
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -124,15 +183,18 @@ namespace bgfx { namespace hlsl
|
||||
|
||||
bx::write(_messageWriter, &messageErr, "Error: Unable to open D3DCompiler_*.dll shader compiler.\n");
|
||||
return NULL;
|
||||
#endif // BX_PLATFORM_LINUX || BX_PLATFORM_OSX
|
||||
}
|
||||
|
||||
void unload()
|
||||
{
|
||||
#if BX_PLATFORM_WINDOWS
|
||||
if (NULL != s_d3dcompilerdll)
|
||||
{
|
||||
bx::dlclose(s_d3dcompilerdll);
|
||||
s_d3dcompilerdll = NULL;
|
||||
}
|
||||
#endif // BX_PLATFORM_WINDOWS
|
||||
|
||||
s_compiler = NULL;
|
||||
D3DCompile = NULL;
|
||||
@@ -214,7 +276,7 @@ namespace bgfx { namespace hlsl
|
||||
const UniformRemap& remap = s_uniformRemap[ii];
|
||||
|
||||
if (remap.paramClass == constDesc.Class
|
||||
&& remap.paramType == constDesc.Type)
|
||||
&& remap.paramType == constDesc.Type)
|
||||
{
|
||||
if (D3D_SVC_MATRIX_COLUMNS != constDesc.Class)
|
||||
{
|
||||
@@ -222,7 +284,7 @@ namespace bgfx { namespace hlsl
|
||||
}
|
||||
|
||||
if (remap.columns == constDesc.Columns
|
||||
&& remap.rows == constDesc.Rows)
|
||||
&& remap.rows == constDesc.Rows)
|
||||
{
|
||||
return remap.id;
|
||||
}
|
||||
@@ -249,9 +311,10 @@ namespace bgfx { namespace hlsl
|
||||
ID3D11ShaderReflection* reflect = NULL;
|
||||
HRESULT hr = D3DReflect(_code->GetBufferPointer()
|
||||
, _code->GetBufferSize()
|
||||
, s_compiler->IID_ID3D11ShaderReflection
|
||||
, s_compiler->reflectionGuid
|
||||
, (void**)&reflect
|
||||
);
|
||||
|
||||
if (FAILED(hr) )
|
||||
{
|
||||
bx::write(_messageWriter, &messageErr, "Error: D3DReflect failed 0x%08x\n", (uint32_t)hr);
|
||||
@@ -427,6 +490,11 @@ namespace bgfx { namespace hlsl
|
||||
bx::strCat(profileAndType, BX_COUNTOF(profileAndType), profile);
|
||||
|
||||
s_compiler = load(_messageWriter);
|
||||
if (NULL == s_compiler)
|
||||
{
|
||||
bx::write(_messageWriter, &messageErr, "Error: Unabled to load D3D compiler!\n");
|
||||
return false;
|
||||
}
|
||||
|
||||
bool result = false;
|
||||
bool debug = _options.debugInformation;
|
||||
@@ -519,7 +587,7 @@ namespace bgfx { namespace hlsl
|
||||
}
|
||||
|
||||
printCode(_code.c_str(), line, start, end, column);
|
||||
bx::write(_messageWriter, &messageErr, "Error: D3DCompile failed 0x%08x %s\n", (uint32_t)hr, log);
|
||||
bx::write(_messageWriter, &messageErr, "Error: D3DCompile failed 0x%08x `%s`\n", (uint32_t)hr, log);
|
||||
errorMsg->Release();
|
||||
return false;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user