Performance of : many variables VS hashes/array with many elements

[azdesign 0]

Hello again everyone,

While I was coding my script, I was wondering that there are many of these variables that called, assigned, disposed, etc across the program. Let me give you a simple example :

 

@offset_top
@offset_right
@offset_bottom
@offset_left

print @offset_top
@image.x = 640 - (@image.bitmap.width + @offset-right )
#etc..

 

VS

 

@offset = {
'top' => 10,
'right' => 10,
'bottom' => 0,
'left' => 10,
}

print @offset['top']
@image.x = 640 - (@image.bitmap.width + @offset['right'] )
#etc..

 

obvious variable name are made to be easier to remember/accessed, while I think, put them into hashes make them tidier (like a database). I like hash/array, centralized similar attributes. Lets assume that there are hundreds of those variable or hash elements, the hash may even have nested elements in the script, which one "lighter" for CPU to call them ?

 

To make it simple, is puts @container['identifier'][index] use significantly greater CPU Usage than puts @Grandparent_Parent_Child ? Is it significant ? or less noticed ? or even almost nearly the same ?

 

I did my own test using animation calling nested hash elements (around 50 elements) per frame but cpu usage (amd athlon II X4 635 @2.9GHz) just went random, sometimes high (around 25), sometimes low (around 13), so I can't get accurate information.

 

Please share me your opinion

[kellessdee 48]

You're going to be hard pressed to get a precise answer and it highly depends on what you are doing with the objects in question.

 

Generally speaking, Array will have better performance than Hash, while a single variable would probably have better performance than both arrays and hashes.

 

An array covers a contiguous block of memory, so you get the advantage of uniform memory access (it is easier to retrieve/set elements) whereas a Hash will generally store it's elements in different memory locations. A Hash also has the overhead memory of it's keys [it needs to store those keys somewhere] -- however you can mitigate that overhead slightly by using symbols for keys, rather than strings (only one instance of any symbol will ever exist in memory, strings will be duplicated in memory)

 

A variable, being a single memory location, is going to have better access performance, because you don't have the overhead of a container object--don't forget that an array/hash variable still requires a variable lookup (to access the hash/array) plus whatever logic is necessary for retrieving the element.

 

However, generally speaking as well--the difference in performance is probably very minimal. I wouldn't worry about performance characteristics until it's necessary. Premature optimization might get you into trouble (and be more work than it's worth). What you SHOULD do, is simply choose the best model for your requirements then usually the performance will be optimal (assuming everything else is optimal)

 

Another thing to note: I did say performance of the various objects depends on the scenario, to give an example:

let's say you have a set of records that contain some data, and you want to find the object associated with a specific name:

# Array
record = nil
for d in data
 if d.name == some_name
   record = d
   break
 end
end
# Hash
record = data[some_name]

 

As you can see, with the array, associating an object with another object forces you to do all the heavy-lifting of looking up that value. Using a hash on the other hand (assuming your keys are the objects that should associate with other objects), allows you to leverage the hash as a lookup-table--you let the hash do all the heavy-lifting.

 

But of course, like I said, worry about picking the CORRECT model for your program FIRST, then worry about optimizing that model AFTER. Also, ONLY worry about optimizing that model IF you need to. If your program/game performs well (no large frame drops, etc.) then you shouldn't need to try to optimize your code. Indeed, you could make your code slower if you aren't careful. After the code is created, then check for slowdowns and try to understand the underlying cause of the slowdowns. You'll probably get better performance optimizing your algorithms, rather than trying to optimize your data structures.

[azdesign 0]

I understand. Before I went to do scripting with ruby, I work with XML, Java, PHP and MySQL and because I accustomed with storing data in structural manner (XML, MySQL), its just become a habit. RPGMaker XP did not allow me to create some kind of custom database other than weapons, items, skills, etc. That is why I put class data in a hash/array as lookups instead. In the meantime, I'll just follow your suggestion, focus on the the algorithm, data and optimization comes later.

 

Is there some kind of built-in modules in RPGMaker, that allow us to work with XML ? Or, did I wrong about that we cannot add another "table" other than the pre-defined ones such as weapons, items, skills, etc ?

 

Thanks

Edited June 25, 2012 by azdesign

[kellessdee 48]

RPG Maker's RGSS library doesn't have anything xml based, I'm sure you could find a free+open source ruby XML parsing library (make sure it's ruby based, native extensions don't work in RPG Maker)

 

Or, if you want something more uniform, you can construct your own data classes in the same way RPG Maker does. It would be similar to defining a table and adding records to that table in MySQL.

 

Let's say you want to record some client data (contrived example)

in SQL you might do [please excuse my SQL syntax, it's been a while]

CREATE TABLE client (
 id numeric,
 name varchar,
 phone varchar
);
INSERT INTO client (
 id,
 name,
 phone
)
VALUES (
 1,
 'Person',
 '1234567890'
);

 

In ruby you'd do something like this:

class Client
 attr_accessor :id
 attr_accessor :name
 attr_accessor :phone
 def initialize
id = 0
name = ""
phone = ""
 end
end

# RPG Maker style data storage:
$data_clients = [] # array
c = Client.new
c.id = 1
c.name = "person"
c.phone = "1234567890"
$data_clients[c.id] = c

 

The good part about using Ruby classes/objects to store your data, is that Ruby has built-in facilities for reading/writing these objects to the disk:

 

# write data
file = File.new("clients.rxdata", "wb")
Marshal.dump($data_clients, file)
file.close
# read data
file = File.new("clients.rxdata", "rb")
$data_clients = Marshal.load(file)
file.close

 

Then you could theoretically create your data outside (or inside) RPG Maker, and use it that way, no XML required.

[Moonpearl 32]

I did not actually run tests on this but in my opinion, it's not really about variables, but rather about objects. I've read an interesting article about that, which said the following.

 

Suppose you wish to read lines from a text file and then write them again (supposedly after some processing). The obvious way is to make a huge string which concatenates each line read from the file. Alternatively, you could put each line into an array and then write each string from the array in sequence. Both solutions require the same number of objects, that is, a String object for each line which is read, plus an additional String object in the first case, or an Array object in the second. The difference, though, is that the former case takes up much more memory, because the huge string contains a duplicate of each line contents, while the latter case's array just features references to those strings. Of course the strings read from the file shouldn't be kept in memory after they've been copied to the huge string since they're not useful anymore, but you cannot explicitly destroy objects, thus leaving them there until the Garbage Collector gets rid of them.

 

So, even if it does not address your issue directly, my best guess is that you should think in an object-wise perspective. For example, I saw you using strings as keys in a hash - well then you'd better use symbols (:top instead of 'top'), because whenever you write a string in your code (whether to write the 'top' key or to access it), a new String object is created, thus leaving you with multiple instances of 'top' in memory, while :top always calls for the same unique Symbol object.

[ForeverZer0 44]

Ruby sucks in the fact that it doesn't offer much control over memory management. Lower level languages like C, C++, hell even C# allow the use of pointers and explicitly freeing resources. Ruby on the hand does its own thing. There is no "passing by reference" in Ruby, everything is passed as a value. What it does instead of modifying the object directly, it always creates a new object. Ruby also offers no way to control where memory is stored, whether on the stack or on the heap. Ruby simply stores everything on the heap. The only real way around this is using symbol for strings, etc, but that is in no way full control, and cannot be used for everything obviously.

 

These "automatic" features of Ruby are both a blessing and a curse. The plus side: it allows for a language that is very simple to learn and provides a great introduction to OO programming. The downside: it lacks performance to do much more than simple processing. 2-D gaming is about as much as it can handle, and only doing that acting as a wrapper for a lower-level library.

 

All in all, Ruby is a language that basically has a "don't worry about that stuff, I'll handle it" kinda of demeanor. In the example you listed above, neither way is going to have any real impact on anything. A struct would probably be the best way to handle it if you were determined to have a container for the values you could reference by name.

 

Offset = Struct.new(:left, :right, :top, :bottom)
offset = Offset.new(0, 3, 0, 3)
right = offset.right

 

If you were after absolute performance at any cost, then a simple 4 element array of integers without any labels is going to offer the best performance. Remember that Ruby arrays cannot perform optimally whenever they are very large. If you have an array that contains hundreds or thousands of objects, then more than likely a hash is going to the best option. RGSS also comes with the Table class, which is a built-in C/C++ class that has high performance containing lots of data. The drawback is that it can only contain 16-byte integers, which aside from that drawback itself, care must be taken, since Ruby's Fixnum class has a wider range than a 16-bit integer.

 

Anyways, I'm rambling now, so hopefully you picked up something from all that nonsense... ;)

[azdesign 0]

Thanks for the input everyone, I gain many things I didn't know before

Firstly, I will definitely try everything like, store data in a class then export/import to a file, symbol in hashes, storing data in Table and using Struct, really thanks for the input, there are so many alternatives I could use (I didn't realize these are exist, well, the tutorials I've been reading only covers so little)

 

And then, I will use the appropriate model for each algorithm in my script. And of course, I will make sure it works first before trying to make it tidy or optimize it.

 

Thanks guys

Edited June 26, 2012 by azdesign