Watch out for the range library compilation times

Ranges

 The ranges library are a great addition to the C++ standard. However they can increase compilation times; especially when used in template headers which are included by other translation units. It also caused the following error: 'error C1128: number of sections exceeded object file format limit: compile with /bigobj'. 

 The specific use case was a template class which used the range library in two ways:

• use of ranges functions; e.g. std::ranges::adjacent_find(...)
• use of a view; e.g.  std::ranges::binary_search(std::views::keys(deq), h);

It turned out that especially the view was heavy for the compiler. Leaving out the range in the code improved compile times with 30% for a piece of code and 10% overall in a large application. The numbers can o.f.c. vary per use case.

Conclusion 

 If compile times are paramount and you have a header which is included a lot and uses the range library you could measure its impact on compilation times. if it contributes too much replace it with pre range constructs.

Installing through IAssemblyCache

Windows Side-by-Side

 When you want to share your DLL's across applications there are a couple of options:

• put in a directory and add a path variable
• dump in the Windows system directory
• put in the Windows Side-by-Side (WinSxS)
• since Windows 7: using application config files and probing. This is the same as the first option but without a fragile global PATH variable. 

 Under Windows XP the first two options were not recommended. The third option was then the way to go but the documentation is not updated. Therefore one may question if it is still recommended.

To put something in the WinSxS one need:

• a manifest
• a signed cat file which lists all the installed components
• one or more DLL's to install 

The following steps were executed:

1.  Create a certificate in PowerShell with e.g. 'New-SelfSignedCertificate -Type CodeSigningCert -Subject "CN=SxSTestCert" -CertStoreLocation "Cert:\CurrentUser\My"'. This creates a SxSTestCert in the personnel section of the certification store. One can check this with 'cermgr.msc'.
2. extract the public key and store in a *.cer file.
3. create a publicKeyToken with SDK tool pktextract.exe from the just created *.cer file. Use calculated value for in the manifest file. This is a crucial step: with an incorrect publicKeyToken one get all kinds of errors.
4. create a manifest (see below). The entries in the manifest file described the side by side assembly
5. create a cdf file listing all files in the manifest file
6. create a cat file from the cdf; e.g. 'makecat.exe Test.cdf'
7. sign the cat file with the certificate with signtool. For example use 'signtool.exe sign /s My /n "SxSTestCert" /fd SHA256 /t http://timestamp.digicert.com Test.MyCompany.MyAssembly.cat' 

 A manifest file may look like:

 <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <assemblyIdentity 
        type="win32"
        name="Test.MyCompany.MyAssembly"
        version="1.0.0.0"
        processorArchitecture="amd64"
    publicKeyToken="91cb7a3ae2229a15"/>

<!-- Add your DLL file here -->
  <file name="MyPayload.dll">
  </file>
</assembly>

The associated cdf file may look like this:

 [CatalogHeader]
Name=Test.MyCompany.MyAssembly.cat
ResultDir=.
PublicVersion=1
CatalogVersion=2
HashAlgorithms=SHA256

[CatalogFiles]
<HASH>Test.MyCompany.MyAssembly.manifest=Test.MyCompany.MyAssembly.manifest
<HASH>MyPayload.dll=MyPayload.dll

Now that you have all files in code you have to invoke 'InstallAssembly'. Note that there is no sxs.lib to link against so the function 'CreateAssemblyCache' must be extracted from 'sxs.dll' through LoadLibrary / GetProcAddress.

#include <winsxs.h>
#include <atlbase.h>

void AssemblyCreate()
{
   CComPtr<IAssemblyCache> ptrCache; 
   HMDULE hModule = ::LoadLibrary(sxs.dll);

   using PfCreateAssemblyCache = HRESULT (*) (IAssemblyCache**, DWORD); 

   PfCreateAssemblyCache pfCreateAssemblyCache = reinterpret_cast<PfCreateAssemblyCache>(::GetProcAddress(hModule, "CreateAssemblyCache"));

   HRESULT hr = pfCreateAssemblyCache(&ptrCache, 0);
      
   FUSION_INSTALL_REFERENCE ref = { 0 };
   ref.cbSize              = sizeof(FUSION_INSTALL_REFERENCE);
   ref.dwFlags             = 0;
   ref.guidScheme          = FUSION_REFCOUNT_OPAQUE_STRING_GUID; // Required!
   ref.szIdentifier        = L"MyTestInstallerApp";
   ref.szNonCannonicalData = L"Test Installation";

   constexpr wchar_t szManifestFilePath[]  = L"Test.MyCompany.MyAssembly.manifest";
   
   hr = ptrCache->InstallAssembly(0, szManifestFilePath, &ref);

   if (SUCCEEDED(hr))
   {
   }
   ::FreeLibrary(hModule);
} 

 This creates a side by side package:

• on Windows 10 on C:\Windows\WinSxS and Manifests folders
• on Windows 11 on C:\Windows\WinSxS\Fusion folder

Acknowledgement

 AI tooling Gemini guided me through the process of uncovering most of the information presented here. A couple of times I was steered in the wrong direction as well. For example on Windows 11 side-by-side components are installed in the Fusion subdirectory which was thought to be a .NET only location by Gemini. It suggested an alternative route through IAssemblyCacheItem but this yielded the same outcome. Only later I had to correct Gemini that Fusion is probably a Windows 11 thing. Claude first classified it as an outdated technology; later recommended this option based on the requirements until it concluded that installing in SxS would not be possible (based on SxS tracing) and only reserved for Microsoft themselves.

Watch out for open plan offices

 

Open plan office

 I had to work for more than a decade in an open plan office which was far from ideal. Open plan offices in the Netherlands are actual open; not with cubicles like in the USA which would give some kind of privacy. There is a lot of visual and auditory noise in the open plan office which makes concentrating and doing your work difficult. Also when you need a meeting to discuss topics you have to reserve a room instead of just talking in your own room.

 The bad part was that I anticipated all this and reported this to my manager when he came up with the initiative to create an open plan office in the new building. He shove the arguments aside and said that I should first try it out before having an opinion. I countered that the old situation was already like an open plan office so I don't need new experience. He also used the dumb argument that he himself worked happily on an open plan office so I should be happy too. He then stated that if it wouldn't work out he would reverse the situation which of course never happened.

 He then put all developers in a big open plan office but he reserved for himself a nice private room. 

 I complained about the open plan office every assessment but nothing changed. When the open plan office was finally under scrutiny the manager even lied that the whole open plan office idea was on our request and now we suddenly wanted something differently. 

 He eventually got fired; not for his open plan office but actually for the wrong reasons. Still the performance drop due the open plan office would have been a strong argument to let him go. A new manager came and put me in a private room. Other developers weren't so lucky: they still had to endure the open plan office.

Careful with refactoring

Refactoring issue

 Last year we applied a small refactoring in a piece of code. The construct was a parent - child relationship with the child hold by unique_ptr. Simplified the code was somewhat as follows:

struct Parent
{
   explicit Parent(bool bShow)
   : m_bShow(bShow)
   {
      m_ptr = std::make_unique<Child>(this);
   }

   std::unique_ptr<Child> m_ptr;
   bool                   m_bShow; 
};

struct Child
{
   explicit Child(Parent* pParent)
   : m_bShow(pParent->m_bShow)
   {
   }

   bool  m_bShow; 
};

 Although the code has a bidirectional dependency between parent and child iIt will compile if one splits out in header and source files for parent and child. 

 The heap use of child seemed redundant so it was removed and the child will be hold by value:

struct Parent
{
   explicit Parent(bool bShow)
   : m_child(this)
   , m_bShow(bShow)
   {
   }

   Child   m_child;
   bool    m_bShow; 
};

 This refactoring lead though to a bug where sometimes things were shown and sometimes not. The bug only appeared in release mode under certain conditions. It turned out that the value of 'm_bShow' was using uninitialized memory. We accidentally created a memory safety issue in years. 

 The problem is that the child is created before the parent is fully created and thereby using uninitialized memory of the parent. The solution is easy by reversing creation order:

struct Parent
{
   explicit Parent(bool bShow)
   : m_bShow(bShow)
   , m_child(this)
   {
   }

   bool    m_bShow; 
   Child   m_child;
};

 Using the 'this' pointer in constructor is a code alarm but not a code smell. As a general rule here declare first all members of built in types before declaring members with classes. If that is not viable one can defer creation by using std::optional instead of std::unique_ptr. std::optional is often more optimal from a performance perspective.

 Not sure how Rust would have prevented this; probably by disallowing the construct in the first place. That would be pity since circumventing the extra heap allocation and pointer access is certainly worthwhile the final solution where children are hold by value.

Watch out for std::vector::at()

 Aspects of operator[] vs at() 

 In order to bump the default memory safety of C++ the committee has decided to harden the STL with adding bounds checking to operator[]. This is redundant since bounds checking is already present through function 'at()'. A safety profile could enforce use of 'at()' and issue a warning for the use of operator[].

 This hardening decision is not a free lunch and has consequences for performance. If we compare current operator[] which has no bounds checking with 'at()' with bounds checking it is 5 times slower in a test case. Below is the test case with two functions:

int g_iTemp = 0;

void PrfStlVectorIteratorIndex(const std::vector<int>& rv)
{
   int nTemp = 0;
   
   const size_t nLoop = rv.size();
   	
   for (size_t n = 0; n != nLoop; ++n)
   {
      nTemp += rv[n];
   }
   
   g_iTemp = nTemp;
}
   
void PrfStlVectorIteratorIndex(const std::vector<int>& rv)
{
   int nTemp = 0;
    
   const size_t nLoop = rv.size();
    
   for (size_t n = 0; n != nLoop; ++n1)
   {
      nTemp += rv.at(n);
   }
    
   g_iTemp = nTemp;
}

The results for a certain test with VS2022 17.14.23 with /O2: 

Function                  #            Total(s) 
PrfStlVectorIteratorIndex         1     0.149972
PrfStlVectorIteratorIndexAt       1     0.727781 

The function using 'at()' is 5 times slower. A reason could be found when looking at the generated assembly. MSVC uses SIMD instructions in case of operator[] but it cannot use them with 'at()'.

Conclusion 

 This is a significant difference. It makes one wonder why the C++ committee took the decision so lightly to tax every invocation of operator[]. Especially since a major use case for operator[] is to use it in a loop as above where there is no danger of going out of bounds. The committee's argument is that it costed only 0.3% extra performance which clearly contradicts above numbers. Also they stated that on certain code bases it revealed thousand extra bugs. Not sure what that code base is. For decades we use Visual Studio with Microsoft's STL which has the extra checking turned on in debug mode and it never fires these asserts when using or testing debug builds. If it would fire it would reveal a bug and one can repair it. Let users who value safety over performance use the 'at()' variants but leave the operator[] alone.

 

Thoughts on C++ 26

Sutter's video

 The other day I watched Sutter's YouTube video about 3 cool things in C++ 26:

1. Make C++ safer by replacing undefined behavior (UB) with erroneous behavior (EB)
2. Reflection
3. Yet another syntax for async

Safe C++ 

Sutter mentions two aspects:

• uninitialized local variables will be data mangled. The compiler may inject code to check if uninitialized variables are accessed.
• hardening of STL; most notably operator[] 

According to studies the overhead is minimal (0.3%). This number is debatable: they can never know what applications are out there. In the past we had bad experience with VS 2008 who turned on safe iterators in release builds. They killed all compiler optimizations right away when used.

I question also the first bullet: why not make it simpler and state that every variable will be default or zero initialized. There is no EB or UB necessary; or no hidden code injected by the compiler.

Some of the hardened STL functions are unnecessary. There are already 'at()' functions which bounds check. A safety profile could warn for use of operator[].

Reflection

Nice that reflection is added but I wonder if the C++ committee has the right priorities. The standard library even lacks a standard JSON or XML library which would be an ideal candidate for automatic serialization through reflection.

Async

They added a new superfluous new syntax. So much for consistency.

 Conclusion

 Memory safety is an issue these days even if so to silence the Rust religion. The C++ committee should do something though I believe more in safety profiles than changing the language fundamentally. Even so I would go then for zero initialization instead of checks with hidden costs. Reflection is nice but what C++ lacks most is standard libraries; not major language changes. Still for us as desktop application developer memory safety is not much of an issue.

Issues with Linux port

 

Linux port

 The company who employs me has decided to port parts of our application to Linux. At a first shot we will use WSL and Visual Studio but issues are not over:

• CMake is the lingua franca of generating cross platform build scripts. CMake is a beast of itself however and not sure why this got so popular.
• WSL keeps sometimes its old configuration and source files. It seems that when the source files are read only they are read only on the target WSL system as well. If you edit a file afterwards the update of this file will fail so one won't see their changes. Either make the source files not read only beforehand or clean all files and directories on the WSL host and then start fresh again. Alternatively make the source files writable on Linux through chmod. 
• MSVC uses __declspec(dllexport) to export functions from DLL's; GCC doesn't have that.
• MSVC is pushing security enhanced versions of the CRT through its code analyzer. According to cppreference these functions are standardized albeit as extension (i.e. Annex K of C11). Unfortunately glibc has not implemented them; partially because some dubious reasoning. The API isn't perfect and there are pre-exisiting bounds checking crt but still it's standardized and some people (we) use them. So one ends up writing Windows and Linux specific code even in a layer which supposed to be platform independent.
• Many of the MSVC C API is Windows specific (e.g. _splitpath; _makepath; _tchdir). Using C alone on Windows platform may therefore still not platform independent.
• GGC bugs:
• Warning suppression's in pre-compiled header in GCC are ignored in code. This is quite unhandy; especially since some suppression's one want to apply globally to all sources and are therefore are primary candidate to put in pre-compiled header. I have filed a bug 123287 report and it's stated that this has been solved for GCC 15.x.
• std::chrono::parse does not support wchar_t.
• erroneous message about possibly dangling reference to a temporary. Fixed in GCC 14.3
• GCC can't handle init of boost::uuids::uuid with a 16 element std::uint8_t array. Fixed in GCC 15.2.

• On Windows wchar_t are 2 bytes and represent UCS-2 or UTF-16. On Linux wchar_t it is 4 bytes and probably represents UTF-32. To be compatible with existing persistence storage we had to use char16_t on certain places in the code. Unfortunately it seems that some character code conversion facilities are deprecated so this solution will not hold out for long.
• std::basic_ifstream and std::basic_ofstream don't accept std::wstring as function name on Linux. This seems to be a MSVC extension so change code to use std::filesystem::path which is a conformance improvement.
• __FUNCTION__ is an extension which both MSVC and GCC understand. On GCC it is not a macro so prepending it with 'L' to get the wide character variant does not work. It also only gives the function name without class in case member function which makes in unattractive. So specific MSVC and GCC code is needed. There is a standard: __func__. However again it only gives the function name. source_location is another alternative but this gives too much information for the function name. 
• std::prev and std::advance with negative offset on GCC increment the iterator in case it's an input- or forward iterator. This lead to a crash with 'std::prev(umap.cend())' since on GCC std::unordered_map iterators are of forward iterator category type while on MSVC they are bidirectional. Also using std::views::transform can result in an input iterator category which would fail then with std::prev. See bug 122224. Preferably a compile time error should be issued but GCC author thinks otherwise. Decided not to use std::prev.
• no leading zero mask in date time (e.g. the'#' in '%#d') is a MSVC extension. Resolution: write ourselves.
• swprintf requires '%ls' for wchar_t arrays. This is unfortunate for writing char / wchar_t agnostic code.

GCC might still contain some basic bugs. The warning about #pragma once in main file (when building a pre-compiled header) is only solved in version 14.