AMD Stream Processing

(Feb 2, 2008)

AMD recently announced its first stream processing board. Although it tried to promote and talked only about its flagship, AMD Firestream 9170 with 320 stream cores, from documents related to CAL requirements, we need only ATI Radeon HD2400 or better. This is good, but I'm sure that not many documents are around, and there must be some unseen issues waiting for me if I jump into stream processing stuff.

I think I should wait a year or two so that the platform will be matured.

Reference: http://ati.amd.com/technology/streamcomputing/

We should try to reduce initialization task ourselves

(Jan 30, 2008)

I tried implement an efficient way to build an isotropic image. For a long time, I thought a compiler optimization will handle initialization overhead automatically, but for Visual C++ 2005, it does not.

Consider the following code:

for( int z = 0; z < y =" 0;" x =" 0;"> arVoxels[8];
NVImage_t arValues[8]; // keep H.U. values of relevant voxels


In fact, we can move all declarations outside the most outer loop (z) as follows:

float fX, fY, fZ; // for true position
float fa, fb, fc; // relative voxel distance as in 2.1.
NVVoxel arVoxels[8];
NVImage_t arValues[8]; // keep H.U. values of relevant voxels

for( int z = 0; z < y =" 0;" x =" 0;" z =" 0;" y =" 0;" x =" 0;"> arVoxels[8];
NVImage_t arValues[8]; // keep H.U. values of relevant voxels

We can see that for each z, there will be two threads. Hence, if we move declarations to the most outer loop, two threads will use the same variable storage, and that is not correct. What we should do is moving these declarations just one step outer as shown below:

#pragma omp parallel
for( int z = 0; z < y =" 0;"> arVoxels[8];
NVImage_t arValues[8]; // keep H.U. values of relevant voxels

for( int x = 0; x < nNewSizeX-2; x++ ) {

From my experiment, this makes things much faster since array initialization is expensive. Note: we can even reduce initialization further if we divide the loop ourselves and create 'multiple sections'. For each section, declare its own variables. I, however, will not do this since the code will be clutter, and we have to know the number of sections before hand, which is normally equal to the number of physical cores in a system. Thus, changing CPU may cause some performance issue. Using OpenMP in this way, nonetheless, is very flexible regarding to system configurations.

Associativity in Modern CPU Cache

(Jan 30, 2008)

Recently, I have used OpenMP to perform multi-processing a lot. However, OpenMP may cause 'false share' often if we are not careful. False share is a situation that two threads write in a different memory locations, but unfortunately, the two memory locations are assigned the same cache slot. If this situation happens, performance will be degraded significantly.

Cache associativity will play an important role on this issue, especially if we have 8 cores or more.
So, let's look at cache associativity for some modern CPUs.

AMD Athlon 64 X2 has 2-way associative L1 cache and 16-way for L2 cache (ref).
AMD Phenom has 2-way associative L1 cache, 16-way for L2 and 32-way for L3 cache s (ref-page 4)

Intel Core 2 E4000 and E6000 series: 8-way associative L1 cache (ref-page 9) and from what I got from CPU-Z, it has 16-way associative L2 cache.

Intel Core 2 E8000 series: 8-way associative L1 cache and 24-way associative L2 cache (from CPU-Z).

So, I think CPUs from both manufactures should do well in scaling, but from what I got from Tom's hardware, Phenom scales very well and better than Core 2 Quad. I, however, cannot confirm this until both platform are more matured and more serious evaluation are available.

หัดใช้ SSE

(Nov 23, 2007) (Thanksgiving Night)

Today, I tried using vector instructions (SSE2 to be specific) in Visual C++ 2003. There are something worth noting:
1. we might always want to use _mm_malloc( data_size, 16 ) in place of new operator to make sure that our dynamically array will be aligned at 16 bytes (this is a restriction for many operations in SSE2; otherwise we will get memory exception). More details can be found at http://www.x86.org/articles/sse_pt3/simd3.htm and http://www.tacc.utexas.edu/resources/user_guides/intel/c_ug/linux117.htm

Remember that, to free the allocated memory, use _mm_free.

2. For those using Visual C++, they have alternatives when they want to aligned data. For example,

__declspec(align(16)) float m_fArray[ARRAY_SIZE];

and 
m_fArray = (float*) _aligned_malloc(ARRAY_SIZE * sizeof(float), 16);

Please see more info at http://www.codeproject.com/cpp/sseintro.asp

3. As can be seen from 2, it is desirable to align our floating-point array as well since this will allow us to cast-and-use the floating-point array.

4. There are interesting classes which are suitable for vector instructions: vector3D and 4x4 Matrix. Please see http://www.x86.org/articles/sse_pt3/simd3.htm (near the page bottom).

More may be available at http://www.codeproject.com/useritems/SSE_optimized_2D_vector.asp (see the source code package).

5. Examples in using add, mul, and sqrt via SSE: http://www.codeproject.com/cpp/sseintro.asp

6. memcpy and _mm_loadu_ps will play an important role to increase speed for convolution since SSE needs to align data at 16 bits, but we want to move convolution window little by little (4 bytes). However, if we interleavedly process data, we need to perform memory move / copy only four times. This should be good for performance improvement.

There is a website about fast factor 2 resampling using SSE at http://mail.gnome.org/archives/beast/2006-March/msg00001.html. I don't know if it does show something related to my convolution problem, but I will one day take a look at it seriously.



7. Very simple example of SSE in GCC: http://www.tuleriit.ee/progs/rexample.php