Very old news (2004)

 

30 November 2004 After a long, long wait, version 0.3.2 of the DSpatial kernel was released on Sourceforge today. While minor in the versioning there are, however, very many changes to the kernel that affect the stability of the kernel as well as extend kernel functionality. There are essentially three major changes:

Source handlers are no longer running in a separate thread. Instead, all access to public methods is protected by using a critical section. This makes the code more transparant, because source handler methods can now be called directly instead of sending a message to the thread. This means that source methods are now not waitable anymore, but this is offset by a newly introduced mehanism to isolate lengthy processes. See below.

A full set of data access methods for raster band data has been implemented in the DSpModuleBase unit. These include blob access (a large rectangular area of data somewhere in the band) and filtered access (a defined, square area of data around a central pixel). Analytical processes use either one of these mechanisms to request data from the kernel, process it, and write results back to the kernel. These processes can run within the main thread, or in a separate thread, requiring only a few extra lines of code. See the Analysis page for more details, including a how-to on developing your own analysis routines.

DSpatial now has its own file format. While we developers dreaded this day, we are nevertheless quite proud of the format. The entire file is organized in tiles of 16kB, making access efficient both for disk access and for storage in main memory. Currently only raster data can be stored, but plenty of it. A single file supports an unlimited number of rasters, that each can have up to 1,024 bands of any (combination of) data type. In the future other data types will be manageable with a DSpatial file as well. The format was developed because of the limitations of other file formats. In a DSpatial file it is possible, for instance, to store names of raster bands (only other format that supports that is HDF), projection information, and metadata. The DSpatial file has now replaced the temporary file format used so far. A word of caution is in place here though: The file format is still not extensively tested and it may change in the near future. Therefore we advise you not to store any critical data in a DSpatial file yet. A word of advise as well: While DSpatial is perfectly compatible with the Delphi memory manager, performance is degraded by the weird alignment of data (a 16kB block can start at any address, instead of on an OS-optimal $1000 boundary). We therefore recommend to use the RecyclerMM replacement. Unfortunately, this replacement memory manager is not available for Kylix.

These changes to the kernel mean that you have to uninstall any previous version of DSpatial you might have, including the visual components and the HDF driver if you have it.

Aspect of Ahern Pass, MT, USAIn addition to these main developments in and around the kernel, there are many additional updates and bug fixes. It is now possible to delete raster bands, for instance (although this is known to cause AVs, we're looking into that). New drivers have been added for ESRI and USGS ASCII grid files. More substantially, the first release of the terra module is made available today as well. Four functions are currently provided by the module: slope, aspect, planiform curvature and profile curvature. These can be derived from any 16, 32, or 64 bit raster band. See the Terra module page for more details.

Logging is now also more useful with a handy log window added. Just press Ctrl-L to view log messages streaming in. Note that you need to enable higher log levels in the DSpatial.inc file. Anything over LOG3 will degrade performance, with LOG5 and LOG6 grinding down your app to uselessness.

Documentation of DSpatial and the modules continues to be absent, even though this web site has had some developer information added to it. This will be the main focus of developer time for the coming month, or so. Other than that, you can expect binary operators in the gridmath module, cost surface functions in the terra module, and a new demo which will evolve into a desktop raster GIS. On the kernel side, release 0.4 will focus squarely on the support for coordinate system conversion. Stay tuned.

 

Sine transformation of a DEM4 April 2004 Version 0.3.1 of the DSpatial kernel was released on Sourceforge today. This is a maintenance release with no new features, but a truckload of bug fixes. Most of these bugs came to light during the development of release 0.1.2 of the gridmath module, also newly available today.

The new gridmath module release 0.1.2 contains a new unit that implements a fairly extensive array of mathematical functions that operate on a single raster band. These functions include simple arithmetic, trigonometry, power and logarithms, and a few support functions. With this module it is now for the first time possible to manipulate raster data with DSpatial!

Attention is now shifting back to the kernel where a so-called filtered data access object is being developed. This filter maintains a defined region of a raster band and it supports notions such as current pixel and relative pixel addressing. This filter access will greatly reduce the complexity of analysis functions that operate on multiple raster bands, or local functions on a single raster band such as those to be included in the terra and orbit modules.

Documentation of DSpatial and the gridmath module are still sorely lacking. Especially needed right now are detailed documents on programming with the DSpatial kernel and the extension modules, and on how to develop additional raster drivers. A volunteer who could take on this task would be greatly appreciated.

A DSpatial news group has just been added on news.gmane.org. The news group gmane.comp.gis.dspatial.devel is a news mirror of the mailing list on SourceForge. The news group has been added to lower the effort to discuss DSpatial issues, using your news reader of choice. More news groups will be added on GMane when the need arises.

 

25 February 2004 DSpatial is celebrating its first anniversary today. Time for a new release!

Version 0.3.0 was released today on Sourceforge. While there is not much change on the surface, a number of quite important changes to the kernel and kernel access have been implemented. First of all, the locking mechanism has completely changed. Locking is now initiated at the "user" level (that is, outside of the kernel), where the kernel generates a token suitable for the requested access. The application needs to explicitly drop the token when done. This dramatically reduces the number of locks that needs to be made, because an application can access multiple data set attributes or perform multiple operations before dropping the token. On the down side, a badly written application can lock a data set indefinitely. Secondly, the kernel is now cross-platform compatible. Compiler directives have been added to enable compilation on D5, D7 and Kylix (anybody any experience with D6?).

The most important change is the improved memory management. It is now possible to access data sources of arbitrary size, and the kernel will automatically limit the amount of data that is held in memory. This is controlled with the FMaxDataSetSize member of the DSpatial object. You can change this size in the constructor of the TDSpatial class in the DSpGlobals unit. Any data set that is smaller than the indicated value will be held in memory in its entirety, which provides the greatest access speeds. Large data sets that would quickly use all virtual memory are only partially loaded, and this is completely transparent for the application developer. This design lays the foundation for the gridmath module and other data access mechanisms, which will be provided in an upcoming release.

The gridmath module also sees the light of day with this release. Unfortunately it is still a very meagre offering, with only a few identity operations available. These are necessary to use the SDTS format drivers. On the up side, the development effort is now focused on this module and incremental updates are expected on a regular basis.

The documentation of DSpatial is also starting in earnest now. Attention is given to a detailed document on programming with the DSpatial kernel and the extension modules, and on how to develop additional raster drivers.

In case you're wondering what this image here might be: it's a logic programming error. Sometimes there is beauty in failure. For more screenshots, check out the 2003 very old news page.

 

10 January 2004 Version 0.3 is well underway, although it will still be a while before it can be released. This upcoming release will see a number of quite important changes to the kernel and kernel access. You can check the status of these and other changes from our Code status page. Version 0.3 is scheduled for release sometime in February, but the CVS already contains many of the new features.

First of all, the locking mechanism has completely changed. Locking is now initiated at the "user" level (that is, outside of the kernel), where the kernel generates a token suitable for the requested access. The application needs to explicitly drop the token when done. This dramatically reduces the number of locks that needs to be made, because an application can access multiple data set attributes or perform multiple operations before dropping the token. On the down side, a badly written application can lock a data set indefinitely.

Secondly, the first analysis functions on rasters will be made available. Basic functions (reading and writing of raster data bands, data type conversion) will be part of the kernel, while basic arithmetic on raster bands will be provided through the gridmath module, which will be released together with kernel release 0.3.

Thirdly, the kernel will be cross-platform compatible. Compiler directives have been added to enable compilation on D5, D7 and Kylix (anybody any experience with D6?). If you want to get started with DSpatial on Kylix right away, you can now download the code from the CVS at SourceForge. Note that the HDF format driver does not yet work with Kylix.

Lastly, user level data access will be more sophisticated, where the application can request specific areas of a raster band, or use a filter that represents a number of rows in a raster band and which also supports relative pixel addressing.

On the administrative side, we welcome Fred Edberg as a new member of the developers team. Fred has extensive and broad experience with Delphi, particularly in database access, statistics, and image progressing. Fred is responsible for the development of the orbit module for image processing.

 


Go to SourceForge
Last modified: 20 July 2005. Page maintained by pvanlaake