Re: How can I minimise data transfer when rsyncing subsequent dumps?

[Steve Drangus]

Peter Brawley Wrote:
-------------------------------------------------------
> > I optimistically thought that any tables with
> autoincrementing
> > primary keys should dump in a deterministic way.
>
> Agreed that's optimistic---doesn't it assume
> there've been no changes since the inserts?
>
> To get your row-by-row differences, wouldn't you
> need two-way exclusion joins on paired tables, and
> wouldn't that be more expensive (by far) than just
> dumb dumping the data?

Well rsync handles the diffing between the two dumps so no need to do exclusion joins.

The key here is that I'm not updating old rows; they are historical and don't change. So if I'm dumping a large table with an autoincrementing primary key, and we're dumping in order of the primary key, then the dump should be identical every time up until we hit a chunk that has new records. But, for whatever reason that's now how it goes. Just trying to nail down the reasons why.