[ViewVC] Annotation of: cvs/deliantra/server/doc/historic/Developers/key-value

Fields that the rest of loader.l doesn't understand are recorded in the
object as 'key/value fields'. The first word on the line is the key, and the
rest of the line (leading and trailing whitespace is stripped) is the value
(which may be an empty string!).

These allow arbitrary addition of new fields to the object definition
without any changes to the loader/saver, and without any increase in object
space.  Use these to reduce the overloading of various fields for sets of
infrequently used objects.

Storing as key/value pairing makes it so that it is only parsed once during
loading - this makes accessing data much faster than if it was just a free
block of text that needed parsing each time something might need to get a
key.  The data itself does remain as text, so if the data is desired in
another format, function will need to convert it (eg, atoi) as needed.

How to use them:
=================

From inside crossfire-server (these are defined in object.c):

const char *get_ob_key_value(object *op, const char *key)

  Returns the value parameter for the given key, or NULL if the
  object doesn't have such an extra_field.  The string returned is
  a shared string so should not be modified.

  Note there is no way to tell if the field does not exist or has
  been set to an empty value.  If this differentiation is necessary,
  a value should be stored away.

int set_ob_key_value(object *op, const char *key, const char *value, int add_key)

  Sets the value for the given key. If add_key is true, it will add the key
  if not defined, otherwise, it only updates existing keys.  Returns
  true/false based on success.

  The passed in value is converted to a shared string.  Thus, once this function
  is called, modifying value should not be done.

  Note - usually add_key should be false - there should be little reason
  to only update existing keys - if you want to store that data away, you should
  do it regardless of the type.

  Passing in NULL as the value effectively clears the value.


Use in objects/maps:
--------------------

Just add the lines to the object, eg
foo bar

will automatically be handled so that key 'foo' is set to 'bar'


When to use key/value compared to adding new fields:
====================================================

The following should be considered:

1) Access to key/value fields is slower than accessing a field directly.
   Thus, should not be used in time sensitive areas (combat related
   basically)

   For example, if the only time the key/value would need to be used is by
   player activating the object, that doesn't happen all that often, so good
   option.
  
   If however the key/value would be examined for map updates (say
   glow_radius) probably not a good option.

   Note also that the performance impact of key/values depend on the
   number of key/values used for the specific object.  If an object has
   10 key/value pairs, access is much worse than if it just has one.

   However, in all cases, performance is slow because a the key has to
   looked up in the shared string database and then pointer comparisons
   done.

   This performance consideration is only something that needs to be if the
   actual key/value is used in the time critical area, and not the object
   as a whole.  For example, setting a key/value for a monster isn't
   a problem if that key/value isn't examined during combat (maybe
   something related to conversation).  However, storing attack
   related value there would not be a good idea.

2) Use of key values is more memory intensive than just adding fields
   for objects.  On a 32 bit system, eg key/value structure is 12 bytes,
   compared to just 4 bytes for a pointer to a string.

   One big advantage of the key value is this memory is only used when
   a key is set.  So if you do the math, if more than about 1/3 of the
   items would have that key/value pair, actually uses less memory to
   just add a field.

   As of this writing (August 2005) there are 30 values used by less then
   10 archetypes.  Some number of these should perhaps be moved to
   key/value lists.

3) Key/value fields shouldn't be used if they are related to an existing
   field.  For example, if there is a 'foo' field, add you want a 
   maxfoo value, it should be done as another field and not as a 
   get_ob_key_value(op, "maxfoo") - trying to main that is difficult.

Caveats:
=========

- key/values don't guard against real fields being used: 
  set_ob_key_value(ob, "hp", "acidic")
  will not have the desired effect.

  The load/save code is such that it will save the key/value lists
  before actual field values.  When loading, the last value is
  used, so in the above example, there may be two lines:

  hp acidic
  hp 20

  After load, the value of hp will be 20, since that is the last value
  loaded.

- Using get_ob_key_value(...) with a key for an actual field in the object
  will not work.  Eg, get_ob_key_value(op, "hp") will not return the
  hp value of the object - there is no easy way for the functions to
  to access all the fields.

- Thanks to its partial dependence on the loader, Crossedit will correctly
  save and load extra_fields. It can't set or change them, however; there's a
  whitelist in Attr.c::allowed_variables.

-  Since all lines are valid, catching errors in arches is now difficult.
  For now, there is a debug statement when we get key/values.  When
  key/values get more use, this will be removed.

- Key/values are limited to single lines - you can't do something like

  startfoo
  line1
  line2
  endfoo

  and have it work.  However, the line length allowed in the loader is
  quite long.

- There is no function that walks the key/value list for you. If this
  is needed, should write your own.  However, sucn functionality really
  shouldn't be needed.

- key names with no value are an indicator to the loader that there is
  no value, and that key should be deleted.
  This is most often used by the saver to denote that the object
  has cleared the key/value.  Eg, the archetype has a key/value
  pair, but this object doesn't have a key/value that name.

- Checking two objects against each other can be costly with key/values -
  its basically a O(n^2) operation because have to check of objects 1
  keys are in object 2 and vice versa.  At a cost of a higher load time,
  if the fields were sorted, this could be much quicker, as logic is then
  becomes iterating both lists at the same time, and if any mismatches,
  know it is different right there.  However, this is likely to only
  become an issue if the number of key/value pairs for any objects go
  above some certain amount.  It's also conceivable since the real
  high cost here is the comparisons, hashing the keys or values and
  storing that hash in the object could be a big gain, but once again,
  depends on how often the values change (as you'd have to recalcuate
  the entire hash)
Revision:	1.1
Committed:	Thu Sep 7 21:42:56 2006 UTC (17 years, 10 months ago) by pippijn
Branch:	MAIN
CVS Tags:	rel-2_82, rel-2_81, rel-2_80, rel-3_1, rel-3_0, rel-2_6, rel-2_7, rel-2_4, rel-2_5, rel-2_2, rel-2_3, rel-2_0, rel-2_1, rel-2_72, rel-2_73, rel-2_71, rel-2_76, rel-2_77, rel-2_74, rel-2_75, rel-2_54, rel-2_55, rel-2_56, rel-2_79, rel-2_52, rel-2_53, rel-2_32, rel-2_90, rel-2_92, rel-2_93, rel-2_78, rel-2_61, rel-2_43, rel-2_42, rel-2_41, HEAD
Log Message:	Moved documents to doc/historic
#	User	Rev	Content
1	pippijn	1.1	Fields that the rest of loader.l doesn't understand are recorded in the
2			object as 'key/value fields'. The first word on the line is the key, and the
3			rest of the line (leading and trailing whitespace is stripped) is the value
4			(which may be an empty string!).
5
6			These allow arbitrary addition of new fields to the object definition
7			without any changes to the loader/saver, and without any increase in object
8			space. Use these to reduce the overloading of various fields for sets of
9			infrequently used objects.
10
11			Storing as key/value pairing makes it so that it is only parsed once during
12			loading - this makes accessing data much faster than if it was just a free
13			block of text that needed parsing each time something might need to get a
14			key. The data itself does remain as text, so if the data is desired in
15			another format, function will need to convert it (eg, atoi) as needed.
16
17			How to use them:
18			=================
19
20			From inside crossfire-server (these are defined in object.c):
21
22			const char get_ob_key_value(object op, const char *key)
23
24			Returns the value parameter for the given key, or NULL if the
25			object doesn't have such an extra_field. The string returned is
26			a shared string so should not be modified.
27
28			Note there is no way to tell if the field does not exist or has
29			been set to an empty value. If this differentiation is necessary,
30			a value should be stored away.
31
32			int set_ob_key_value(object op, const char key, const char *value, int add_key)
33
34			Sets the value for the given key. If add_key is true, it will add the key
35			if not defined, otherwise, it only updates existing keys. Returns
36			true/false based on success.
37
38			The passed in value is converted to a shared string. Thus, once this function
39			is called, modifying value should not be done.
40
41			Note - usually add_key should be false - there should be little reason
42			to only update existing keys - if you want to store that data away, you should
43			do it regardless of the type.
44
45			Passing in NULL as the value effectively clears the value.
46
47
48			Use in objects/maps:
49			--------------------
50
51			Just add the lines to the object, eg
52			foo bar
53
54			will automatically be handled so that key 'foo' is set to 'bar'
55
56
57			When to use key/value compared to adding new fields:
58			====================================================
59
60			The following should be considered:
61
62			1) Access to key/value fields is slower than accessing a field directly.
63			Thus, should not be used in time sensitive areas (combat related
64			basically)
65
66			For example, if the only time the key/value would need to be used is by
67			player activating the object, that doesn't happen all that often, so good
68			option.
69
70			If however the key/value would be examined for map updates (say
71			glow_radius) probably not a good option.
72
73			Note also that the performance impact of key/values depend on the
74			number of key/values used for the specific object. If an object has
75			10 key/value pairs, access is much worse than if it just has one.
76
77			However, in all cases, performance is slow because a the key has to
78			looked up in the shared string database and then pointer comparisons
79			done.
80
81			This performance consideration is only something that needs to be if the
82			actual key/value is used in the time critical area, and not the object
83			as a whole. For example, setting a key/value for a monster isn't
84			a problem if that key/value isn't examined during combat (maybe
85			something related to conversation). However, storing attack
86			related value there would not be a good idea.
87
88			2) Use of key values is more memory intensive than just adding fields
89			for objects. On a 32 bit system, eg key/value structure is 12 bytes,
90			compared to just 4 bytes for a pointer to a string.
91
92			One big advantage of the key value is this memory is only used when
93			a key is set. So if you do the math, if more than about 1/3 of the
94			items would have that key/value pair, actually uses less memory to
95			just add a field.
96
97			As of this writing (August 2005) there are 30 values used by less then
98			10 archetypes. Some number of these should perhaps be moved to
99			key/value lists.
100
101			3) Key/value fields shouldn't be used if they are related to an existing
102			field. For example, if there is a 'foo' field, add you want a
103			maxfoo value, it should be done as another field and not as a
104			get_ob_key_value(op, "maxfoo") - trying to main that is difficult.
105
106			Caveats:
107			=========
108
109			- key/values don't guard against real fields being used:
110			set_ob_key_value(ob, "hp", "acidic")
111			will not have the desired effect.
112
113			The load/save code is such that it will save the key/value lists
114			before actual field values. When loading, the last value is
115			used, so in the above example, there may be two lines:
116
117			hp acidic
118			hp 20
119
120			After load, the value of hp will be 20, since that is the last value
121			loaded.
122
123			- Using get_ob_key_value(...) with a key for an actual field in the object
124			will not work. Eg, get_ob_key_value(op, "hp") will not return the
125			hp value of the object - there is no easy way for the functions to
126			to access all the fields.
127
128			- Thanks to its partial dependence on the loader, Crossedit will correctly
129			save and load extra_fields. It can't set or change them, however; there's a
130			whitelist in Attr.c::allowed_variables.
131
132			- Since all lines are valid, catching errors in arches is now difficult.
133			For now, there is a debug statement when we get key/values. When
134			key/values get more use, this will be removed.
135
136			- Key/values are limited to single lines - you can't do something like
137
138			startfoo
139			line1
140			line2
141			endfoo
142
143			and have it work. However, the line length allowed in the loader is
144			quite long.
145
146			- There is no function that walks the key/value list for you. If this
147			is needed, should write your own. However, sucn functionality really
148			shouldn't be needed.
149
150			- key names with no value are an indicator to the loader that there is
151			no value, and that key should be deleted.
152			This is most often used by the saver to denote that the object
153			has cleared the key/value. Eg, the archetype has a key/value
154			pair, but this object doesn't have a key/value that name.
155
156			- Checking two objects against each other can be costly with key/values -
157			its basically a O(n^2) operation because have to check of objects 1
158			keys are in object 2 and vice versa. At a cost of a higher load time,
159			if the fields were sorted, this could be much quicker, as logic is then
160			becomes iterating both lists at the same time, and if any mismatches,
161			know it is different right there. However, this is likely to only
162			become an issue if the number of key/value pairs for any objects go
163			above some certain amount. It's also conceivable since the real
164			high cost here is the comparisons, hashing the keys or values and
165			storing that hash in the object could be a big gain, but once again,
166			depends on how often the values change (as you'd have to recalcuate
167			the entire hash)