[#CORE-2519] Do not per default serialize byte[] for BLOB columns using Object(Input|Output)Stream - DataNucleus JIRA

JPOX Core (ARCHIVED)

Do not per default serialize byte[] for BLOB columns using Object(Input|Output)Stream

Created: 09/Sep/05 10:35 PM Updated: 19/Sep/06 02:49 PM Resolved: 13/Jan/06 08:32 AM

Component/s:

Datastore Support

Affects Version/s:

1.1.0-beta-4

Fix Version/s:

1.1.0-beta-6

Description

« Hide

In RDBMSManager line 152, it reads:

registerDatastoreMapping(byte[].class.getName(), BlobRDBMSMapping.class, "BLOB", "BLOB", false);

The use of BlobRDBMSMapping results in ObjectInputStream/ObjectOutputStream being used to "mangle" the data of the byte[] when it is mapped to a BLOB column. This kind of mapping cannot be used when reading existing binary data, e.g. image data in JPEG format that had been written simply as the series of bytes of a .jpg file.

Either BlobRDBMSMapping.getObject(Object rs, int param) should be changed to do the same as Blob2RDBMSMapping.getObject(Object rs, int param), or RDBMSManager.initialise() should be changed to favour Blob2RDBMSMapping instead of BlobRDBMSMapping.

Sort Order:

[ Permalink | « Hide ]

Andy Jefferson added a comment - 10/Sep/05 11:01 AM

It is like that because it works for the *vast majority* of RDBMS. If some RDBMS (MSSQL maybe ?) needs something else then it goes in that RDBMS' manager.

[ Permalink | « Hide ]

Joerg von Frantzius added a comment - 11/Sep/05 07:50 PM

This happens with any DB using the default mapping set out in RDBMSManager for byte[] and BLOB columns.

The point is that ObjectOutputStream seems to add some bytes to the byte[] before writing, and ObjectInputStream expects this header to be able to read. Possibly no-one has encountered a problem with this yet, but might very well do so in the future when it turns out that the bytes in the column are not just the bytes of the byte[] written into it. You can only read this again using a Java ObjectInputStream, any other access e.g. using a different language, will be very difficult.

That's not an issue of MSSQL, but of the use of ObjectOutputStream and ObjectInputStream in BlobRDBMSMapping.

Blob2RDBMSMapping uses PreparedStatement.setBytes() and ResultSet.getBytes(), which doesn't alter the byte data.

[ Permalink | « Hide ]

Andy Jefferson added a comment - 07/Jan/06 06:12 AM

This shows up on the storage of serialised BigDecimal fields in the TCK. What we should do is actually simplify the options :-)
We have the mapping for the field within the BlobRDBMSMapping/LongVarBinaryRDBMSMapping. This tells us if the field should be serialised. Consequently we should just have BlobRDBMSMapping and LongVarBinaryRDBMSMapping that perform serialisation if the field needs it, and otherwise use the byte[] stream (as found in Blob2RDBMSMapping now). Make sense ?

[ Permalink | « Hide ]

Andy Jefferson added a comment - 07/Jan/06 08:17 AM

JPOX CVS now only serialises the field if it is either marked as "serialized", or if it is a type that has no support for byte streaming (so it isnt an array, and isnt BitSet). Try this on MSSQL and see if it does what you want. I'll mark it as resolved if I hear nothing within a few days

[ Permalink | « Hide ]

Andy Jefferson added a comment - 13/Jan/06 08:32 AM

Assumed fixed.

[ Permalink | « Hide ]

Robert Wetzel added a comment - 19/Sep/06 02:49 PM

I've stumbled into the same problem today, after some codecrawling it turns out "serialized" is overwritten internally, even though it has been explicitely put into the meta data info as "false" for the specific class. As a result it tries to put the byte[] data into a stream and fails.

Seems like it runs into the following code part of org.jpox.metadata.AbstractPropertyMetaData (line ~1000), of jpox 1.1.2:

        if ((hasCollection() && SCOUtils.collectionHasSerialisedElements(this)) ||
            (hasMap() && SCOUtils.mapHasSerialisedKeysAndValues(this)) ||
            (hasArray() && SCOUtils.arrayIsStoredInSingleColumn(this)))
        {
            // Collection/Array with serialised elements or Map with serialised keys and values so set as serialised for later
            serialized = Boolean.TRUE;
        }