[Additional Exercises][File
Processing Performance][Win32 vs. C Run-Time Library][Comments on Win32 API][Windows CE][Windows NT vs. UNIX][Pthreads
Emulation][Additional Programs][Windows
Me][Windows XP][POSIX Queues][About the Author]
DOWNLOAD Version 1.10 (January 28, 2008) (about 1.7 MB) from here. Principal change: Chapter 12's SendReceiveSKST.c, with the projects rebuilt. Also, Chapter 10 has an improved version of the ThreeStage.c program written for Pthreads. See the ReadMe.txt file for additional changes.
Note to Edition 2 and Edition 1 readers: There has been some chapter rearrangement; see the readme.txt file. I've made number of corrections and have added a few additional examples since Editions 1 and 2. Projects are in both Microsoft Visual Studio .NET and Microsoft Visual C++ Version 6.0 form.
Additional Note for Visual Studio 2005 users: VS2005 will convert the VS .NET solutions when you open the solution.
Additional Note for Vista Users: I have not tested every program. However, experience to date indicates that programs run fine under Vista, as one would expect. If you find any problems, please send an email message!
Edition 3 Readers: Edition 3 has attempted to correct all defects and issues identified with Editions 1 and 2. Nonetheless, most of the comments below and links at the top of the page apply to all editions.
March 31, 2007. Visual Studio 2005 and other VS versions. I've used several VS versions with the sample projects and have not had any difficulties except you will be prompted to convert the project. Please contact me if you run into any problems or have any comments.
March 31, 2007. Vista. I've run a few programs under Vista without difficulty (which is what you'd expect). However, I'd be happy to hear from readers regarding any comments, problems, successes, etc.
June 5, 2005. Projects7 (Visual C++ 7.0 projects directory). Several projects were set up to build to the run6 directory rather than run7. This is fixed in the 1.4 code download. Affected projects are: asc2unDLL, JobShellSecure, sortMM (debug build only), and Utility_3_0 (debug build only). Thanks to Bill Stanton for finding this error.
Chapter 1, p. 16, bullet 4. March 31, 2007. Aaron Yang points out an important misstatement. fread and fwrite return the number of objects, not the number of bytes, read or written. This does not affect Program 1-1 because the object size is one.
Chapter 2. May 16, 2005. I mention here and in Appendix C that FILE_FLAG_SEQUENTIAL did not seem to help performance significantly. However, I recently chatted with a colleague who has processed large files (a gigabyte or so) and has found a significant benefit from this flag. I'll try to pin down more information.
Chapter 2, p. 28. April 17, 2006. YangYang, from ChengDu, China, points out that creating a file with TRUNCATE_EXISTING fails if the file does not exist, contrary to the statement in the book. Experiments quickly confirm this fact, which is also stated in MSDN. Furthermore, this is the expected behavior, considering the option's name.
Chapter 2, p. 32. April 17, 2006. Bullet 4, Line 3 (5 lines from the bottom of the page). It says "_tstcpy (for strcpy)". This is incorrect. and should say "_tcscpy (for strcpy)".
Chapter 2, p. 42, June 7, 2005. Program 2-2. I've used ReportError() throughout. However, you may prefer to use the standard strerror() function to generate text error strings.
Chapter 2, p. 48. May 16, May 28, 2005. The reference to BY_HANDLE_FILE_INFO is incorrect; it should be BY_HANDLE_FILE_INFORMATION. Also, MOVEFILE_WRITETHROUGH, used with MoveFileEx, should be MOVEFILE_WRITE_THROUGH. Thanks to Paul Narula (narula@math.utexas.edu) for identifying this mistake.
Paul pointed out an omission in the MoveFileEx discussion on p. 48., for which I apologize and thank Paul. In his words:
"If dwFlags specifies MOVEFILE_DELAY_UNTIL_REBOOT and lpNewFileName is NULL, MoveFileEx registers the lpExistingFileName file to be deleted when the system restarts." http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/movefileex.asp Initially I thought that you might have figured out that the delay until reboot flag wasn't really necessary so I wrote a quick C program to test that theory. I called MoveFileEx("c:\\delete_me.txt", NULL, 0) and then I rebooted the machine. The file didn't get deleted. But when I called MoveFileEx("c:\\delete_me.txt", NULL, MOVEFILE_DELAY_UNTIL_REBOOT) and rebooted, the file did get deleted. I really don't think this is a big deal since I would imagine that most people who read the book have a copy of the MSDN Library at their disposal. Anybody who read the MSDN documentation would know immediately what they needed to do. But if somebody was using only the book they might not understand why the file remained on reboot." JH Comment: This is an interesting and important detail, and it would have been good to mention it in the text. Nonetheless, there is a great deal of detail in MSDN, and it can't al be in text, which is not intended to be a reference. Deciding what to put in and what to omit was a judgement.
Chapter 3, pp 59-60. Dec 19, 2004. The references to the function SetValidFileData are incorrect. There is no such function; the correct name is SetFileValidData. Thanks to Simon Bailey for finding this error.
Chapter 3, p. 62 (Program 3-1), May 28, 2005. Thanks to Bill Stanton for finding this important defect. In the two QuadPart calculations, it's important to cast at least one of the multiplication operators to LONGLONG to assure 64-bit arithmetic. Casting sizeof(RECORD) is sufficient. Thus, we have lines 82 and 106 of RecordAccess.c changed to:
CurPtr.QuadPart = (LONGLONG)sizeof(RECORD) * atoi(argv[2]) + sizeof(HEADER); and
CurPtr.QuadPart = (LONGLONG)RecNo * sizeof(RECORD) + sizeof(HEADER);
Incidentally, I completed a project recently that required huge file processing, and everything worked well using code similar to this example (of course, I did the 64-bit arithmetic properly). However, the error in the book did not occur to me until Bill pointed it out.
Chapter 3. p. 69 (Program 3-2, lsW.c), Oct 9, 2005. TempPath is not initialized; it should be initialized the same way that PathName is. This problem is corrected in Version 1.7 of the download zip file. Shin-Wei Hwang reported this problem and the fix.
p. 72 (Program 3-3), Sept. 5, 2005. Thomas Beard points out that the logic for SetAccessTime and SetModTime is reversed. Furthermore, there are some other logic changes to emulate the UNIX version's error reporting (or lack thereof) correctly. The corrected code can be downloaded as Version 1.6.
Chapter 4, pp. 114-115 (Program 4-5). June 23, 2007. A reader points out that the second print statement ("Leaving handler in 6 seconds" ) rarely appears. This is because the main program, running in a separate thread from the handler, samples the Exit flag every 5 seconds and usually exits before the handler can print the separate message. It would be more accurate for the first message to say ""Ctrl-c received by handler. Leaving in 5 seconds or less." and for the second to say that it will be one second or less. Furthermore, it would be best to set Exit in the individual case statements, as you might not want to terminate the program in all situations. The source file (CHAPTR04\CNTRLC.C) is fixed in the downloadable solutions.
Chapter 5, p. 136. Sept 6, 2006.
Eric Berge says, correctly, that there is an error in the description of
OpenFileMapping() on page 136.
The dwDesiredAccess values are not the same
as dwProtect in
CreateFileMapping() but rather should be
the same as the dwAccess values to
MapViewOfFile()." This makes sense as you are specifying
the access that you wish to have, and dwDesiredAccess
is checked against the security descriptor of the named file mapping object.
Chapter 5. Nov. 21, 2004. randfile.c
has been updated so that all generated lines terminate with CR, LF. The files
can then be used with programs such as sortBT.c,
which require data files with CR, LF terminated lines. August 7, 2005. Matthew Wilson, the author of Imperfect C++,
makes an interesting point about memory mapping handles, and I'd neglected this
point in Chapter 5. "Just a quick note about the use of memory mapped
files, specifically wrt the timing of CloseHandle().
A file mapping object maintains a reference to the file which it maps, so
one can call CloseHandle() on the file object as
soon as CreateFileMapping() has succeeded (or
failed, for that matter). Similarly, a view maintains a reference to the file
mapping, so one can call CloseHandle() on the
file mapping object as soon as MapViewOfFile(Ex)()
is called. In this way, one need maintain only a single 'handle' (the void*
pointer to the view) which is especially useful for creating C++ wrapper classes
utilizing simple RAII (as seen in the WinSTL memory_mapped_file class, part of
the STLSoft libraries)." JH Note:
"STL" is the C++ Standard Template Library.
Chapter 7, pp 207-208, and Appendix A, pp 480-481. June 20, 2007. Clive Darke asked about the bottom of page 207, which tells you to #define _MT, as does the Appendix A listing of Envirmnt.h. However, if you use Visual Studio to set the "Mutlithreaded DLL" C++ code generation options, _MT will be defined for you. This is the correct way to define this macro.
Chapter 9, p. 266-268, Sept 5, 2005. Jemzer Ulrich noted that the first three lines on the top of p. 268 are in the wrong order. The return 0; should obviously come last. This applies to statsMX.c, statsCS.c, statsIN.c, and statsNS.c. There is no practical impact, however, as process termination will free all the resources.
Chapter 9, p. 273. June 7, 2005. Bill Stanton pointed out a significant error in the code fragment at the top of the page. The semaphore wait and release should be outside the loop, as in the TimedMutualExclusion example program and as follows:
WaitForSingleObject (hThrottleSem, INFINITE);
while (TRUE) { // Worker loop
WaitForSingleObject (hMutex, INFINITE);
... Critical code section ...
ReleaseMutex (hMutex);
} // End of worker loop
ReleaseSemaphore (hThrottleSem, 1, NULL);
The original form would simply aggravate the situation with additional waits. In this new form, the number of active worker threads contending for the mutex is limited. An alternative, which would still limit the number of active threads but which would be fairer by preventing a single worker from hogging the semaphore, would be the following (I have not tried this in TimedMutualExclusion, but doing so would be an interesting experiment - I'd be interested in hearing from anyone who does try it):
DWORD count = 0, limit = 1000 /* a tunable, arbitrary value */;
. . .
while (TRUE) { // Worker loop
if (count % limit == 0) WaitForSingleObject (hThrottleSem, INFINITE);
WaitForSingleObject (hMutex, INFINITE);
... Critical code section ...
ReleaseMutex (hMutex);
if (count++ % limit == 0) ReleaseSemaphore (hThrottleSem, 1, NULL);
} // End of worker loop
Chapter 9, p. 275. June 7, 2005. Second section ("System, Process, and Thread Affinity Masks", second bullet, should read "The process mask ...". Thanks, again, to Bill Stanton.
December 30, 2005. Program 10-2, Page 290. Shin-Wei Hwang noticed that the CreateThresholdBarrier() function uses an auto-reset event; it should use a manual-reset event, which is appropriate for the "broadcast model". Line -6 should be:
hthb->b_broadcast = CreateEvent (NULL, TRUE /* Manual-Reset. */,
The program on the disk/web site used the signal model, which also works (each release thread signals to release another thread). However, I've change the code to be consistent with the code in the book.
December 30, 2005. Page 293, last sentence of the first full paragraph. Replace the last sentence with, "The first two functions provide placeholder values and logic for time outs, which could be a useful additional feature."
August 14, 2005. Program 10.4, Page 295. Serious error in q_full(). The logic is clearly wrong and has been fixed in the download code. The problem was spotted by some sharp-eyed course participants at Ask Jeeves. The correct code for the function is:
DWORD q_full (queue_t *q)
{
return ((q->q_first - q->q_last) == 1 ||
(q->q_last == q->q_size-1 && q->q_first == 0));
}
It's remarkable that this bug. which reversed the use of the first and last indices, was undetected for such a long period of time - very embarrassing!
Chapter 10, January 28, 2008. David Harding asked about the Pthreads version of ThreeStage.c; I have added an updated version of the code to the download file. You can run it using the open source Pthreads library.
Chapter 11, Figure 11-1. August 14, 2007. Program 1 on the bottom left of the figure contains a typo. The second line should be:
hWrite = GetStdHandle (STD_OUTPUT_HANDLE);
This is symmetrical to Program 2 (P2), so that the pipe is known as hWrite in P1 and hRead on P2. Likewise, P1 takes input from hIn, and P2 sends output to hOut. Thanks to Greg Gagne, coauthor of the Operating Systems Concepts series (Silberschatz, Galvin, and Gagne) who brought this to my attention.
Chapter 11. Program 11-3 (serverNP). March 5, 2005. First, there's a minor difference between the code in the book and on the disk. About 2/3 down on p. 332, interchange the two lines,
CloseHandle(hConTh); and
if (ShutDown) continue;
Also, we need to test hConTh == NULL before caling GetExitCodeThread(). Futhermore, hConTh is not closed, although the system will shut down, so it should not be a big problem.
More seriously, however, is a problem that Scott Drummonds (www.e-scott.net) pointed out. My technique to shut down stray connection threads is, well, a bit of a kludge. As Scott points explains,
"In the example code on page 332, you have the Server() thread wait for the connection thread or a change in value in the ShutDown variable. When the Server() thread is shutdown (through the variable), it attempts to close its Connect() thread by connecting to the named pipe. However, the call to ConnectNamedPipe() in Connect() could connect to *any* Connect() thread. It won't just connect to the one owned by the first Server() thread!
Take a simple example with only two Server() threads, 'A' and 'B'. During shutdown Server() 'A' may kill Connect() 'B'. Server() 'B' will then realize that it has no living Connect() thread and die gracefully. At that, Server() 'A' will wait indefinitely for Connect() 'A' to die.
Now, the code on the disk has several suggested alternatives, but none of them seem quite right. Here's my suggestion, not yet tested (watch this space!), which uses asynchronous I/O, which doesn't get covered until Chapter 14. Here's an idea which allows the connection thread to poll the Shutdown flag. Create the named pipes with FILE_FLAG_OVERLAPPED. Then, modify Connect() to use an overlapped structure (of course, we haven't covered overlapped I/O at this point in the book). Then, in Connect(), you can wait on the manual reset event with a time-out, which allows you to loop and poll Shutdown. Thoughts or comments are appreciated; I've been busy and have not been able to try this out.
January 28, 2008 and May 28, 2006. Program 12.4, Page 368-370.
First, Armando Fortuna and Wangzhi have pointed out that the DLL_PROCESS_DETACH case should free the TLS index:
TlsFree(TlsIx);
before the return statement.
NOTE: I would expect that the TLS index would be freed when the process terminates. However, I haven't been able to find any documentation that specifically says that this happens. Furthermore, it's good, safe, programming practice to free all resources specifically. May 29, 2007. Wangzhi reported that the zip file on the web site was not been updated; there is now a new version with this code change. It's version 1.9. I apologize for the omission.
Next, Jun-Gil Jeong pointed out a problem in the ReceiveCSMessage function, which is now fixed in the download file. The nRemainRecv -= nXfer; statement is not needed before the comment:
/* Transfer to target message up to null, if any */
Chapter 13, Nov. 21, 2004. Severity A. serviceSK.c. See the comments in the source code (version 1.2 download). The checkpoint is now updated reliably and the service is also properly set the the started state. There are several other improvements as well. Thanks to Steve Gibson for identifying the problem and suggesting the soluions.
Edition 3, Chapter 14. See the important note below about atouMT. April 13, 2005 - Problem FIXED. Download code to get the fix; see the notes below for an explanation.
This page contains additional comments, references, information, etc. that will be useful for readers of Win32 System Programming, Second Edition. Reader comments and input are welcome and will be included as appropriate. Please communicate directly to me at jmhart@world.std.com.
Global comment (April 10, 2004). Many utility programs, especially in the early chapters, have executable names that are identical to the Unix commands that they mimic. This may be inconvenient if you use a Unix-like environment, such as the Cygwin (www.cygwin.com) shell, and you should consider renaming the executables generated by some of the projects (example: Chapter 3's ls project creates ls.exe. The output could be renamed to lsW.exe. Likewise, Chapter 2's cat.exe might be renamed to catW.exe).
There is an error in the solution to Program 1-2 (cpW):
Line 13 should be
if (argc !=3) {
as it is on Page 15 of the book.
For additional Microsoft information about Win32, C Library, and other file "handles" and conversion between them, see (thanks to Christoph Dalitz for this suggestion):
http://support.microsoft.com/default.aspx?scid=kb;EN-US;q99173
April 4, 2004. Severity B. Program 3-2 (ls). Stewart Weiss ( stewart.weiss@hunter.cuny.edu ) pointed out that ls does not work properly with path names such as C:, C:\, and C:\dirname. This has been fixed, but path names are not allowed to end in a backslash. Also, C: will be treated as C:\ rather than the current working directory. The fix has also been applied to lsFP in Chapter 5.
April 4, 2004. Severity A - Program 5-3. InitUnFp.c does not properly allocate memory before calling FindGroup at Line 96 (see the disk; the details are not in the book). This has been fixed; download the full zip file (see the top of this page). Thanks to Daniel Jiang ( dxj_tiger@hotmail.com ) for this, and several other, corrections.
April 4, 2004. Severity B. Program 5-2 (lsFP). See the comments on ls in Chapter 3.
April 10, 2004.Severity B. Daniel Jiang provided another very good observation. The access denied ACEs used in chmod will deny SYNCHRONIZE rights, which, in turn will deny all access as the generic read, write, and execute rights masks all use the SYNCHRONIZE bit. The updated source files provide a correction and a pointer to the relevant Microsoft documentation.
April 4, 2004. Severity C (Information only) DLL Information. "depends" is a handy Windows utility to view DLL information. "dumpbin" is also useful for viewing libraries.
April 4, 2004. Severity B. JobShell.c, Line 175 should be:
JobNumber = _ttoi (argv [iJobNo]);
Several comments and range checks have been added to JobShell.c and JobMgt.c, Thanks to Daniel Jiang ( dxj_tiger@hotmail.com ) for these corrections and suggestions.
Daniel also provides some important job object information: "Job objects are briefly mentioned in chapter 7 and are needed for ex 7-6. ... By default once a process is assigned to a job, all its newly created child processes are added to the job, trying to assign any child process to the job will fail. It took me quite some time to figure this out when doing the exercise."
April 20, 2004. Job Shell can be fairly useful, and I recently completed a consulting engagement in which I was able to combine and modify Job Shell along with Chapter 12's client/server system to manage jobs in a network.
Errata. Severity A
Thanks to Asko Kauppi < Asko.Kauppi@fi.flextronics.com > for pointing out the next two problems and the first one in Chapter 11.
Feb. 21, 2002. p. 245. top (Grey box). The function name should, of course, be DeleteCriticalSection().
Feb. 21, 2002. P. 261, small font, middle of the page. The next to last sentence should read, "...and is therefore similar to PulseEvent applied to manual-reset event." Note: This feature will be used in the Chapter 10 examples. The last sentence would be more helpful as "There is no exact equivalent of SetEvent and ResetEvent used with a manual-reset event."
Table 9-1, Top of Page 262. Delete the last sentence in the PulseEvent/Manual-Reset entry. If no threads are waiting, then the event is reset and PulseEvent returns; no threads are released. The similar table in http://world.std.com/~jmhart/comments.htm (go to the second edition, p. 261 entry) does not have this error.
Comment: I am chagrined by this error, which is inexplicable. Thanks to a sharp-eyed reader for pointing it out. Nonetheless, there is no effect on any of the sample programs or the surrounding text. In the Chapter 10 examples, in particular, PulseEvent is always used as part of the "condition variable model," which uses a condition variable predicate and is always tested even after a thread is released from a wait function. As discussed at length elsewhere, this condition variable model, adapted from Pthreads condition variables, is, in my opinion, essential for reliable thread synchronization.
April 4, 2004. Exercise 10-12. There are several corrections to MultiSem.c and TestMultiSem.c. Daniel Jiang ( dxj_tiger@hotmail.com ) was the source for these corrections. Furthermore, MultiSem.c and MultiSemX.c and were reversed.
Using the Queue Object. A while ago (September, 2000), I completed a consulting assignment which used Programs 10-3 and 10-4 almost unchanged. The resulting implementation became a significant part of a larger system. The solution was especially interesting as the client required that the solution be portable to several UNIX (including Linux) platforms as well as Compaq's OpenVMS (formerly Digital VMS); these other platforms all support Pthreads for thread management and synchronization. I was able to write simple thread management and synchronization macros that allowed for the Win32-Pthread portability. The condition variable model discussed in Chapter 10 was the key to getting everything to work properly. I'll post the macros with some sample programs at a future date; contact me directly if you are interested. For more information, go to the end of this page, "Emulating POSIX Message Queues."
Incidentally, another part of the solution required interprocess communication. For that task, borrowed pieces of the socket server in Chapter 12, especially the connection management logic, worked well.
August 14, 2005. Program 10.4, Page 287. Serious error in q_full(). The logic is clearly wrong and has been fixed in the download code. The problem was spotted by some sharp-eyed course participants at Ask Jeeves. The correct code for the function is:
DWORD q_full (queue_t *q)
{
return ((q->q_first - q->q_last) == 1 ||
(q->q_last == q->q_size-1 && q->q_first == 0));
}
It's remarkable that this bug was undetected for such a long period of time.
Debugging Multithreaded Applications (pp. 296 ff). I wrote at length (perhaps too much length) about the perils of relying on debugging and testing multithreaded applications. In order to reinforce the points there, it is worth looking at http://support.microsoft.com/support.kb/articles/Q173/2/60.ASP. This page documents Microsoft debugger defects when dealing with synchronization.
Exercise 10-12. This exercise asks you to implement a semaphore with an atomic multiple wait. The solution provided on the disc is incorrect as it used the signal condition variable model rather than the broadcast model. This would be OK, however, if threads never release more than one unit at a time. The change from the signal to broadcast model is simple and is included here.
There is a LINUX Open Source POSIX Threads for Win32 project underway, and a snapshot was posted as far back as September 8, 2000, and new updates have been posted since then. Experience to date has been very positive (as of Sept. 1, 2002) and continues to be so (April 20, 2004). See http://sources.redhat.com/pthreads-win32 for more information. See (New, Oct 14, 2000) Open Source for preliminary comments, timing results, and some simple macros that achieve much the same thing as the open source solution, although the macros make several simplifying assumptions about how the Pthreads API is used.
For another example as well as another advanced synchronization technique using QueueUserAPC(), see Batons.
Errata - Severity C - February 21, 2002
Figure 11-1, p. 303. Program 2, on the right, might make more sense if the WriteFile (hOut) were inside the loop, with an indication of missing code between the read and the write. On the other hand, it really depends on what the programs are doing. Perhaps Program 2 is processing the input and then putting out summary data. This is, in fact, the case with the way in which I sometimes demonstrate Program 11-1, where Program 2 is a word counting program.
Program 11-3, p. 319. Severity B - March 5, 2005
Download the new code file to see the additional code that would appear at the end of the page. Also, see additional comments on the same program above under Edition 3.
See the comments below under Chapter 14 for an example of how to use sockets with I/O completion ports when building a scaleable server.
Errata - Severity A
April 4, 2004. serverSK, pp. 345 ff, has several serious problems related to the server thread and system shutdown. The original code passed some tests, for a variety of reasons, but I should have been more alert to the actual shutdown behavior. These problems also apply to serverSKHA and serverSKST and have been fixed on the zip file. Thanks to Pat Wacker ( pwacker@zambasolutions.com ) for finding this problem.
(Nov 21, 2004) See the Edition 3 comment above.
The October, 2000 issue of MSDN (formerly Microsoft Systems Journal) has an article that used sockets as the handles for an I/O completion port. This article also shows additional WinSock 2.0 features. See Windows Sockets 2.0: Write Scalable Winsock Apps Using Completion Ports by Anthony Jones and Amol Deshpande, MSDN Magazine, Oct., 2000, pp. 112-121. MSDN articles are posted at http://msdn.microsoft.com/msdnmag/default.asp.
APRIL 13, 2005 - WARNING - Sept 19, 2004. atouMT.c This program is cited in the text and provided with the sample code. I recommended it as alternative model to overlapped and extended I/O. I've recently tested it some more and it looks very much as if the use of multiple handles on the same file is not thread safe, and atouMT.c give incorrect results as well as read and write failures from time to time. Under investigation! Watch this space.
The fix is really very simple, and the problem was due to an elementary mistake (in fact, I'm more than a little embarrassed by the mistake!). Download code version 1.3 for the fix (go to the top of this page for the download). The problem was this:
Incidentally, I just finished a project for a client; the project used techniques from atouTM.c, and it has proved to be very robust. Furthermore, I wrote the code to be single-source for Windows, Linux (RedHat, but it should work everywhere), IBM AIX, and SunOS. In principle, is should be portable to any POSIX compliant platform. Importantly, however, the Windows version DOES NOT use Microsoft's POSIX system (the client did not want to use it) or any open source code (another client limitation). For I/O, I used Window's _open(), _read(), etc. functions and created macro wrappers around them. For thread management, I used the pthread_emulation macros, described at the end of the link.
p. 508, Paragraph in the middle, after the fifth bullet, second sentence, should read, "For example, cpUC (last row), with an average of 7.77 seconds in the fifth column (W2000 Xeon), ...”
p. 516, Bullet 1 Clarification. I tried different time out values and found that 5 ms gave the smallest elapsed time. Then, I used that same value (5 ms) for all the other cases in column 1 (1, 2, 4, 8, 32, 64).
p. 518. "cpTIME.bat" is repeated. Incidentally, there are other timing batch files in the directory.
These questions will test your understanding of the sample programs and point out interesting details in the program logic. Common techniques for using the Win32 API are noted. There are also suggestions as to how to enhance or improve the programs. I have used many of these questions in courses based on the book.
Appendix C shows sequential file processing performance using the ASCII-to-Unicode programs. This section shows the results using files much larger than the 5MB input file used for the Appendix C results. In some cases, the memory-mapped results are not as favorable as in Appendix C. This section also shows results where memory is flushed to the file at the end and where there is no buffering in the file access version.
This information augments the discussion on Page 10 regarding the choice between Win32 and the Standard C Library (SCL) and the more comprehensive C Run-Time Library (CRTL) when performing file I/O.
This section contains a variety of comments criticizing, in a constructive manner, various aspects of the Win32 design (disclaimer: as a whole, I like the API) and implementation. These comments are my own or, in some cases, have come from readers and course participants. Additional reader comments are welcome, as is any information that augments or corrects my statements.
This section contains my thoughts, somewhat dated as of Feb 21, 2002, on this favorite topic. I've also included comments from readers, colleagues, and others.
Many programmers consider the POSIX Pthreads standard API to be a superior threading and synchronization environment compared to Win32. This section discusses open source emulation issues, problems and performance. For additional comments, see the end of Critical Comments.
This section analyzes performance when several threads contend for a single resource. Graphs, with analysis, show performance results under a variety of circumstances, and a test program is included for carrying out additional experiments. Readers have contributed significantly to the test results and interpretation. Critical sections will normally provide superior performance (under NT), but, surprisingly, this is not always the case. Contending for critical sections when running more than four threads or when running in the background can be slower than using mutexes, and performance can degrade dramatically. Furthermore, under Windows 95/98, CS performance is better in the background than in the foreground.
Finally, critical sections provide no performance benefit at all when running on an SMP system and two threads are running on different processors and contending for the same critical section. CRITICAL_SECTION.SMP performance is significantly degraded when using multiple processors in such situations, and you will get better performance by forcing all contending threads on a specific processor using the thread processor affinity mask.
The program in this section can be also used as a simple method of estimating performance of threaded, synchronized code using different synchronization techniques. It also allows you to test the effect of serializing thread execution and using nested CS/Mutex calls, and parameters allow you to simulate the effect of CPU-intensive and I/O-intensive threads. The number of threads concurrently contending for the same resource is another command-line argument.
This section includes a test program, performance results with graphs, and analysis.
A reimplementation of pipe (Program 11-1) using named pipes in byte mode.
This program, contributed by a reader, is similar to the other copy programs in Chapter 1, and it is comparable to the memory-mapped ASCII to Unicode program, atouMM, in Chapter 6. It further demonstrates the speed of memory-mapped I/O and illustrates some alternate coding styles to those I used in the book.
The Chapter 7 job management programs allow you to manage processes that are created under job management. In many cases, however, you may want to find out all the processes running on the system; that is, you would like the equivalent of the UNIX ps command. There is no obvious way to list all the running processes, but take a look at http://www.codeguru.com/win32/ps.shtml for a solution to the problem. Note: I have not tested this; use at your own risk.
Windows Me ("Millennium Edition") is the latest and last (YES! XP obsoletes the entire 9x family. Horray!) release in the Windows 9x series. In theory, every statement in the book about Windows 9x should apply to Windows Me as well, and preliminary testing indicates that this is the case. The July 2000 MSDN library does not differentiate Windows Me from Windows 95/98, and there is not even an index entry for "Windows Me." Timing results are at Windows Me. (April 20, 2003). The Windows 9x family really is almost a thing of the past, although many PCs continue to use Me, 98, and even 95. Microsoft no longer sells 9x systems (since the introduction of XP). Even Windows 2000, and its NT 5.0 kernel, are becoming increasingly rare.
Updated Sept 19, 2004. Windows XP and 2003 are the latest releases in the Windows NT series. Every statement in the book about Windows NT and Windows 2000 apply to Windows XP as well, and testing confirms this. (April 20, 2004). XP (NT 5.1 kernel) and 2003 (NT 5.2 kernel) are all that are currently sold. The NT 5.1 kernel introduced a few new features, such as vectored exception handlers. Synchronization performance appears to be improved in many cases.
The intent of this system is to emulate the four POSIX (Linux and UNIX) message queue
(MQ) functions:
1) msgsnd
2) msgrcv
3) msgget
4) msgctl
on any platform that supports Pthreads as well as Windows. The system has been tested on
Windows (9x, Me, NT, 2000, and XP) and on Compaq OpenVMS (yes, there are still
dedicated users).
The MQ functions allow for message types (hence priorities) within a FIFO discipline, and
each individual MQ is identified by a user-defined "key" that is similar to a
file name. Then, msgget() converts the key to a MQ ID,
which is somewhat similar to a file descriptor. msgsnd()
and msgrcv() are for sending and receiving messages, which
are typed with what amounts to a priority, so that msgrcv()
can extract a message of a specified type or of type below a threshold. msgctl() is used to set attributes of a specified MQ.
Typed message applications include:
The man pages on any UNIX system provide abundant additional information.
The emulation is written so that it will run on Windows as well as on Pthreads
platforms. This is achieved by using macros (see the file thread_emulation.h)
that emulate the required Pthread functionality under Win32. You can also use (with a few
source code changes so as to #include "pthread.h")
the Open Source pthread emulation library, available from (among other places):
http://sources.redhat.com/pthreads-win32/
Also, see my comments on the Open Source pthreads library at:
http://world.std.com/~jmhart/opensource.htm
NOTE: Windows programmers who would prefer not to have the unfamiliar Pthread functions in
the code are welcome to replace the functions with their macro equivalents, as defined in thread_emulation.h.
The component pieces of the emulation, WHICH CORRESPOND EXACTLY TO WINDOWS
PROJECTS AND TO THE EXCECUTABLES OR LIBARIES, are:
Click here to download the code and projects.
There are several built-in limits, all controlled by macros.
Maximum number of message queues: 256
Macro: MSG_MAX_NUMBER_QUEUES in msq_emulation_internal.h
Note: MQs can be destroyed with a msgctl call, so the
limit only
applies to the number of active queue keys at any one time.
Maximum number of bytes in a single queue message: 65K
Macro: MQ_MAX_MESSAGE_SIZE in msq_emulation_internal.h
Maximum total number of bytes in a single queue:
64 * MQ_MAX_MESSAGE_SIZE = 2**22 = 4MB
Macro: MQ_MAX_BYTES_IN_QUEUE
Maximum total number of bytes in all queues:
MQ_MAX_BYTES_IN_QUEUE*MSG_MAX_NUMBER_QUEUES/2 = 2**29 = 512MB
This looks too large! We may need to decrease it
The security features are not properly emulated.
The message queues are only shared by the threads of a process, rather than being
accessible to threads in multiple processes. A correct emulation would required shared
memory; for an example of process shared objects in a Windows environment, see: http://world.std.com/~jmhart/mltwtsm.htm