VB.Net
versus VB6 performance tests
Author:
Jack Hoxley
Written: 9th July 2002
Contact: [EMail]
Download: Complete Archive
(54kb)
Contents
of this lesson
1. Introduction
2. Maths Performance
3. Memory Performance
4. Further Performance Tests
1.
Introduction
As
many of you will be aware already  in February
2002 I wrote a review of Visual Studio
.Net for this website (you
can read it here). In this review I
discussed the performance of the new
compiler, and came to the following
conclusion:
"I
will be happy to hold up my hand and
admit my fault should my values be
proven to be wrong  I still find it
surprising that I got those results, and
would quite like to be proven wrong!
.Net isn't universally slower than the
previous versions, as I have found a few
cases where it is a bit faster, but that
doesn't hold much in it."
At
the time of writing that review I'd only
been using the .Net languages for a
couple of weeks  not really enough time
to become particularly skilled with it.
However, in the months since then I've
done a little more work and a little
more research (thanks to Patrice Scribe
for getting the ball rolling lately!)
and found that there were flaws in the
code I used and that the times quoted
were therefore misleading.
The
main problem with my previous quoted
times was that I was using VB.Net to
program in a VB6 way  I've used VB6 for
4 years now, and because of obvious parallels
I continued writing my code in this way.
This is not a good strategy for using
.Net; to really get the most
(performance and features) out of the
.Net frameworks you can't always program
in the same way as you would have done
before.
So,
it's come to this. I decided to write
this article to set the record straight
 I got it wrong first time around, and
hopefully I won't get it wrong this time
around. There is no way to conclusively
prove and/or test all aspects of
performance, but I shall instead cover a
few examples that I think are
representative of multimedia
programming.
2. Maths Performance
Maths is critical to how a multimedia
program will perform, the reliance on
maths is huge  and therefore even minor
speed differences in this area can have
a significant impact over all. This
section is divided into two sections,
the first analyzing the core maths
functions (those built into the standard
libraries), and the second being more
complex algorithms using these functions
/ performing more complex functions.
2.1 Core Math Functions:
The core math functions to be tested are
shown in the following listing:
'//VB.Net
functions:
System.Math.Sqrt()
'Square
Root
System.Math.Sin()
'Sine
System.Math.Acos()
'Inverse
Cosine
System.Math.Atan()
'Inverse
Tangent
System.Math.Asin()
'Inverse
Sine
System.Math.Log()
'Natural
Logarithm
System.Math.Exp()
'Exponential
(e^{x})
System.Math.Log10()
'Logarithm
to
base
10
'//VB6
functions:
'all
are equivalent
of the
above
list,
but
some
require
addition
derivation
Sqr()
Sin()
ArcCos()
Atan()
ArcSin()
Log()
Exp()
Log10()
'the
following
3
functions
had to
be
derived:
Private Function
ArcCos(A
As
Double)
As Double
ArcCos = Atn(A / Sqr(A * A + 1)) + 2 * Atn(1)
End Function
Private Function
ArcSin(X
As
Double)
As Double
ArcSin = Atn(X / Sqr(X * X + 1))
End Function
Private Function
Log10(X
As
Double)
As Double
Log10 = Log(X) / LogOf10
'made Log(10) a constant. faster!
End Function 



Each of these functions was taken in
turn, and put into a simple framework
looking like that in the following
listing. Each test was iterated 1
million times, and timed using the high
performance counter. This overall time
was then divided by 1 million to get an
average time for each call  in the case
of maths functions these are quoted in
nanoseconds. Input variables for the
maths functions were the loop variables,
using a constant would have given the
compiler too much opportunity to
optimize away the entire test, and using
random numbers proved to be misleading
(the times represented the performance
of the random number generator AND the
math function).
'//VB.Net
framework:
'//Note:
QueryPerformanceCounter()
isn't
in the
.Net
framework.
The
closest
'//possible
is
System.Environment.TickCount(),
but
after
extensive
testing
I
'//decided
I
couldn't
trust
its
accuracy.
bRet
=
QueryPerformanceCounter(i64Start)
For
I = 0 To
nTests
 1
rVal =
Asin(I
/
nTests)
Next
bRet =
QueryPerformanceCounter(i64End)
bRet =
QueryPerformanceFrequency(i64Frq)
rVal =
((i64End

i64Start)
/
i64Frq)
/
nTests
'//VB6
framework:
lRet = QueryPerformanceCounter(iStart)
For I = 0
To nTests  1
rVal = ArcSin(I / nTests)
Next
lRet = QueryPerformanceCounter(iEnd)
lRet = QueryPerformanceFrequency(iFrq)
rVal = ((iEnd  iStart) / iFrq) / nTests 



Both programs were then compiled
using the highest level of optimization
possible (optimize for speed), and
the system was restarted (to eliminate
any overheads). Each program was then
run 10 times to generate a list of times
for each function. For reference, the
test system used was a 700mhz AMD Athlon
Thunderbird with 288mb PC100 RAM and
Windows XP Professional.
The times recorded (and averaged over
the 10 tests) are shown in the following
table:
Test Name 
Visual
Basic 6 Time (ns) 
Visual
Basic .Net Time (ns) 
Square Root 
46.7791 
34.4608 
Sine 
160.4986 
162.5435 
Inverse Cosine 
608.3858 
375.6694 
Inverse Tangent 
259.7525 
286.1751 
Inverse Sine 
405.3237 
401.6973 
Natural Logarithm 
194.9694 
226.1549 
Exponential 
121.0084 
193.8514 
Logarithm to base
10 
209.4735 
230.0629 
As you can see straight
away its a fairly mixed set of results, I've
highlighted the tests in red where .Net
is slower, and in green where .Net is
faster. Currently it's 4 against and 4
for  tiebreak. Three of the tests where .Net wins can
be explained quite easily  they are the
three function that had to be derived in
VB6. This means that there is the
functioncalling overhead to consider,
which will generate some slowdown.
However, if you were to perform some
highlevel optimization of the VB6 code,
you can change the times to look like
this:
Test
Name 
Visual
Basic 6 Time (ns) 
Visual
Basic .Net Time (ns) 
Inverse
Cosine 
435.8120 
375.6694 
Inverse
Sine 
427.40923 
401.6973 
Logarithm
to base 10 
203.6077 
230.0629 
In
doing some rather trivial optimizations
on the code (making them inline, adding
constants where applicable) you can
reduce the times quite significantly
(although Inverse Sine is an exception).
However, whilst inverse cosine (for
example) got 1.4x faster, it was not
enough to catch up with VB.Net's initial
performance value.
2.2
Applied Mathematics
This next section of maths related
statistics covers applied maths  that
is, helper functions / algorithms that
are highly mathematically orientated.
Anyone who's done any 3D graphics
programming (Using Direct3D for example)
will know that in every frame alone
there can be a huge number of
complicated equations and algorithms to
process. These aren't necessarily all
related to using the core maths
functions shown above, but they will
feature.
The
reason for including this section is
that while the core maths functions are
regularly used, it's these real world
examples that are more common, and any
speed difference here is likely to have
a far bigger impact on realtime
performance than anything else.
Creating
a 4x4 transformation matrix
Essential to all 3D graphics is the
transformation matrix. This is typically
constructed using a series of helper
functions (exposed through the D3DX
libraries in DirectX8). A while back (in
this tutorial) I proposed a faster
function for generating a complete
transformation matrix. Basically, the
source code takes in rotation, scaling
and translation parameters and returns a
properly generated 4x4 matrix. The
source code is fairly boring, so I
didn't bother inserting it in the main
article text, but you can find it in the
downloadable archive.
The
times for creating a complete matrix:
Test
Name 
VB6
Times (ns) 
VB.Net
Times (ns) 
CreateMatrix(
) 
1265.5043 
1214.1316 
The
times are still biased in VB.Net's favor,
but the speed difference is much less
than that shown in earlier tests.
However, the difference shown here of 51.3727
nanoseconds would translate into VB.Net
being able to process the function
33,435 times more persecond than VB6
could. Small difference, big result.
Finding
the inverse of a 4x4 transformation
matrix
Inverse matrices aren't used as heavily
as normal transformation matrices, but
they are the only method for doing
certain very useful techniques (such as
'picking' 3D objects based on a 2D mouse
coordinate). Inverse Matrices by their
very nature are quite slow; so this
speed test is crucial.
As
with the CreateMatrix() test, the source
code is not presented here for clarity,
you can find it in the downloadable
archive. Thanks to Eric Coleman for
letting me use his optimized inverse
matrix function.
Test
Name 
VB6
Times (ns) 
VB.Net
Times (ns) 
InverseMatrix(
) 
843.4019 
1009.7311 
The times
shown here are very much in VB6's
favor. This particular function uses
only *, /, + and  operations which
would eliminate any of the performance
issues with core maths functions shown
above.
Normalizing
a Vector
This is a rather trivial piece of code 
but is often essential to many other
vectorbased calculations. This test
represents the normalization of a 3D
vector generated using random numbers in
the interval [5,+5] for each component
(x,y,z). The list of vectors were
generated separately so as not to introduce
any additional variables into the final,
timed, test. The second time represents
an inline'd version of the code  so as
to remove any functioncalling
overheads.
Test
Name 
VB6
Times (ns) 
VB.Net
Times (ns) 
Normalize
Vector (function) 
194.8496 
165.7895 
Normalize
Vector (inline) 
183.5613 
209.2288 
What
is most odd about the tabulated results
above is that VB.Net produces a slower
time for the inline version of the code.
It is traditionally assumed that inline
functions (of a similar, small, size)
will run faster  as is shown by VB6.
Overall, VB.Net has produced the fastest
possible time  165ns is considerably
faster than anything VB6 can offer.
RayTriangle
Intersection Tests
This is a very useful test to perform 
essentially this can be considered as a
raytracing algorithm for triangulated/tessellated
geometry. It can also be used quite
effectively for collision detection
between two 3D meshes. Raytraced
graphics often produce the best
lighting/rendering but cannot as yet be
used in realtime purposes  instead
people often use raytraced light maps
in games.
Test
Name 
VB6
Times (ns) 
VB.Net
Times (ns) 
Ray
Triangle Intersection 
610.4232 
733.7162 
The
code involved in this algorithm is very
complicated  and makes use of several
nestedfunctions. It would appear
though, that VB6 has the upper hand
again.
2.3
Summary
This
next table is a final summary of all the
times collected for the math functions,
this time including a percentage
difference to compare the two compilers.
Test
Name 
VB6
Times (ns) 
VB.Net
Times (ns) 
VB.Net
is... 
Square
Root 
46.7791 
34.4608 
1.36x
faster 
Sine 
160.4986 
162.5435 
1.01x
slower 
Inverse
Cosine 
435.8120 
375.6694 
1.16x
faster 
Inverse
Tangent 
259.7525 
286.1751 
1.10x
slower 
Inverse
Sine 
405.3237 
401.6973 
1.01x
faster 
Natural
Logarithm 
194.9694 
226.1549 
1.16x
slower 
Exponential 
121.0084 
193.8514 
1.60x
slower 
Logarithm
to base 10 
203.6077 
230.0629 
1.13x
slower 
CreateMatrix(
) 
1265.5043 
1214.1316 
1.04x
faster 
InverseMatrix(
) 
843.4019 
1009.7311 
1.20x
slower 
Norm.
Vec. Function 
194.8496 
165.7895 
1.18x
faster 
Norm.
Vec. Inline 
183.5613 
209.2288 
1.14x
slower 
RayTriangle
Intersect 
610.4232 
733.7162 
1.20x
slower 
There
are the final results  VB.Net wins on 5
counts, VB6 wins on 8 counts. Whilst
that does make VB6 the
"winner" in most senses, there
isn't a huge amount in it  VB.Net is on
average 1.034x slower (it performs at
96.7% of VB6's speed). As you'll find later on, you can
get the source code used here for your
own inspection should you not believe
the data presented.
Whilst
this is not the focus of this article,
I've included a quick summary of the
above data but for C++; in particular
VC++6 and VC++.Net compilers.
Test
Name 
VC++6
(ns) 
VC++.Net
(ns) 
VC++.Net
is... 
Square
Root 
98.8819 
36.5847 
2.70x
faster 
Sine 
246.7142 
176.8138 
1.40x
faster 
Inverse
Cosine 
179.9197 
222.6877 
1.24x
slower 
Inverse
Tangent 
96.3011 
79.1565 
1.22x
faster 
Inverse
Sine 
166.4979 
162.1950 
1.03x
faster 
Natural
Logarithm 
204.7946 
190.4992 
1.08x
faster 
Exponential 
171.0973 
150.2244 
1.14x
faster 
Logarithm
(base 10) 
204.6446 
190.3695 
1.07x
faster 
CreateMatrix(
) 
1059.0915 
1048.2232 
1.01x
faster 
InverseMatrix(
) 
843.6558 
826.6082 
1.02x
faster 
Norm.
Vec. Func. 
124.7684 
105.0628 
1.19x
faster 
RayTriangle
Test 
4728.2339 
2084.6925 
2.27x
faster 
Bare in mind that the C++
language is an international standard 
therefore the source code used for each
compiler was identical. The timing
differences here really should be only
down to the differences in compilers.
Given these results it would imply that
the VisualC++.Net compiler is capable of
producing marginally faster code. The
above results show, that on average, the
.Net compiler generates code 1.33x
faster.
3. Memory Performance
Memory
allocation and processing.
Lets take an arbitrary example
whereby a function must allocate a
significant amount of memory and then
perform a process on it. Similar in many
respects to directly accessing the
memory behind a surface or texture in
Direct3D/Draw. The test code looks like
this:
'//DESCRIPTION:
This
sample
simulates
generating
a
simple
pointlight
based
lightmap.
The
source
code
is
highly
optimised
(thanks
to
Eric
Coleman
for
pointing
a few
things
out!).
For
example,
the
use of
square
roots
(for
distance
calculations)
have
been
optimized
away 
giving
a
significant
speed
boost
for
both
languages.
'//VB.Net
Source
Code
Private
Sub
GenerateLightMap(ByVal
LightR
As
Integer,
ByVal
LightG
As
Integer,
_
ByVal
LightB
As
Integer,
ByVal
cX As
Integer,
ByVal
cY As
Integer)
'we
take
this
array
as
being
the equivalent
of a
640x480x32
surface
Dim
Data(640,
480) As
Integer
Dim
X As
Integer,
Y As
Integer
Dim
Dist As
Integer,
Attenuation
As
Double
Dim
pxR As
Integer,
pxG As
Integer,
pxB As
Integer
For
X = 0 To
639
For
Y = 0 To
479
'1.
work
out
the
distance
from
this
pixel
to the
light.
Dist =
((X
 cX)
* (X 
cX)) +
((Y 
cX) *
(Y 
cX))
'if
the
distance
is
within
200px
then
we
consider
it to
be
lit.
If
Dist
<=
40000 Then
'2.
We now
need
to
work
out
the
attenuation
constant
'(I'm
just
gonna
use
standard
linear
interpolation)
Attenuation
= 1 
(Dist
* 0.000025)
'3.
We now
need
to
work
out
what
colour
this
pixel
will
be.
pxR =
LightR
*
Attenuation
pxG =
LightG
*
Attenuation
pxB =
LightB
*
Attenuation
'4.
we now
need
to
pack
the
colours
into a
32bit
ARGB
Data(X,
Y) = (pxR
*
65536)
Or
(pxG *
256) Or
pxB
End
If
Next
Y
Next
X
End
Sub
'//VB6
Source
Code
Private
Sub GenerateLightMap(LightR
As
Long, LightG
As
Long, _
LightB
As
Long, cX
As
Long, cY
As
Long)
'we take this array as being the equivelent of a 640x480x32 surface
Dim Data(0 To 639, 0 To 479)
As Long
Dim X
As
Long, Y
As Long
Dim Dist
As
Integer, Attenuation
As Double
Dim pxR
As
Long, pxG
As
Long, pxB
As Long
For X = 0
To 639
For Y = 0
To 479
'1. work out the distance from this pixel to the light.
Dist =
((X  cX) * (X  cX)) + ((Y  cY) * (Y  cY)))
'if the distance is within 200px then we consider it to be lit.
If Dist <=
40000
Then
'2. We now need to work out the attenuation constant
'(I'm just gonna use standard linear interpolation)
Attenuation = 1  (Dist
* 0.000025)
'3. We now need to work out what colour this pixel will be.
pxR = LightR * Attenuation
pxG = LightG * Attenuation
pxB = LightB * Attenuation
'4. we now need to pack the colours into a 32bit ARGB
Data(X, Y) = (pxR *
65536)
Or
(pxG *
256) Or
pxB
End If
Next Y
Next X
End Sub 



The
times for the memory allocation and
processing tests:
Test
Name 
VB6
Times (ms) 
VB.Net
Times (ms) 
Memory
Allocation 
39.7021 
17.6280 
VB.Net's version of the
code is 2.33x faster than VB6. This
would indicate that VB.Net, the CLR
and/or the .Net frameworks have a far
superior memory manager.
In
realworld terms, should you wish to use
this process/technique to generate
lighting in your games the two times
listed above correspond to ~24fps and
~58fps (maximum possible). The former
being VB6's time, which while
interactive would prove to be quite
jerky; the latter being VB.Net's which
would be very smooth and pose no real
problem to interactivity.
The
next table shows the C++ compiler
performance for the same code.
Test
Name 
VC++6
Times (ms) 
VC++.Net
Times (ms) 
Memory
Allocation 
28.9813 
21.7283 
4.
Further Performance Tests
The
tests shown in this article can only
represent a small percentage of
realworld applications. For multimedia
work, the best possible test would be to
compare the performance of DirectX
through .Net and VB6  real frame rates
and perframe timing can then be
extracted, and actually mean something.
However,
this test is not going to happen any
time soon. DirectX 9 (at time of
writing) is far from being ready for
release, but does include a managedcode
core, such that it can be properly
tested using the .Net languages. BUT,
DirectX 9 will not support VB6, so it is
impossible to directly compare VB6 and
.Net's DirectX performance. The closest
we can get is to compare DirectX 8 and
DirectX 9  but this adds additional
variables into the system, the biggest
one being that test results will reflect
any difference in DirectX 9 and DirectX
8 and not the actual language
differences.
Ignoring
DirectX for a moment  the difference
between both languages would be best
judged by comparing a much larger
program. One example that I would like
to have done is a landscape raytracer.
This type of algorithm can take upwards
of an hour to complete, and relies
extremely heavily on maths and memory
manipulation. Unfortunately this is a
rather complex program that would take
quite a while to get working properly
and comparably in both languages 
something I didn't have time to do when
writing this article. Maybe in the
future.
As
mentioned throughout this article, you
can download the source code and examine
it in your own spare time  generate
results specific to you computer etc...
The source code is available here,
or from the top of the page.
Finally,
if you think you have significantly
sized code that you think makes a good
comparison (regardless of the overall
results) between VB.Net and VB6 please
email me about it  Jack.Hoxley@DirectX4VB.Com
, even if you don't want to share the
code!
