test size when switching encodings

[butonic]

I'm trying to find a code path that triggers the wrong recalculation of the unencrypted filesize.

In my case I have a 19686944 byte encrypted file on disk but a 19447696 byte size in the db. The difference seems to be the overhead for binary encoded chunks.

bytes on disk = 19686944
encryption block size = 8192

19686944 / 8192 = 2403,19140625
19686944 % 8192 = 1568

So we have 2403 blocks of 8192 bytes and an end block with 1568 blocks.
Now lets try to calculate the size as if binary encoding were used.

header size = 8192 bytes
signature size with padding = 96 bytes -> 8096 unencrypted bytes per block

The first block is the header, so we take one block less to calculate the unencrypted size:
2402 * 8096 = 19446592 unencrypted size (without the end block)

chop off the signature from the end block:
1568 - 96 = 1472

19446592 + 1472 = 19448064

this is 368 bytes  more than the unencrypted size 19447696 in the db

19447696 - 19448064 = 368

hm, where did I miscalculate? It feels too close (in the range of less than 8k / one block) to be incidental. My gut tells me there is a codepath that calls $encryptionModule->getUnencryptedBlockSize($signed) without the encryption module having been initialized first. The encryption.use_legacy_encoding config value may not have been taken into account, causing the default value (false) to return 8096 for the unencrypted block size.

... 3 hours later ...

the size difference in the last block is caused by fixUnencryptedSize using the base64 encoding for the end block as it actually initializez the encryption module:

again, chop off the signature from the end block:
1568 - 96 = 1472

base64 encode:
1472 * 3 / 4 = 1104

19446592 + 1104 = 19447696

19447696 is what is in the DB

So the root cause was indeed a call to fixUnencryptedSize while the encryption module was not already initialized.

This can happen when trying to restore an encrypted file from a backup and running a filescan, setting the encrypted column in the db > 0 and then running the filescan again. Ther may be other codepaths to trigger this.

A might be to always respect encryption.use_legacy_encoding when getUnencryptedBlockSize() is called. BUt I don't know if that causes problems with the default binary encoding ... hm ... something for a test case.

Signed-off-by: Jörn Friedrich Dreyer jfd@butonic.de

[CLAassistant]

All committers have signed the CLA.

[ownclouders]

💥 Acceptance tests pipeline apiMasterKeys-latest-mysql8.0-php7.4 failed. The build has been cancelled.

https://drone.owncloud.com/owncloud/encryption/2584/18

[butonic]

the test needs to set encryption.use_legacy_encoding and then

1. copy an encrypted file and the corresponding key on disk,
2. run a file scan command to pick up the encrypted file from disk (kind of mimicking a backup restore)
3. run a fix versions command
   now the wrong filesize will have been written to the database: see the top post for calculation details.

with owncloud/core#40240 the miscalculation is fixed

[phil-davis]

It is good to check the size of encrypted files as seen by the server.
We could have better steps to avoid having to use a When step like:
user "Alice" gets the size of file "/textfile1.txt" using the WebDAV API
in the Then section of the scenarios.
But those steps are already like that in core, so we can live with a bit of "non-standard" scenario step format here.

[phil-davis]

I will merge this when CI finishes, then make an issue for what other test scenario(s) might be useful to have.