[colug-432] file replication across data centers

[Rick Troth]

[top posting]

/Fabulous/ reply from Angelo.

_I must recommend RSYNC_ as a first course for proving things or for
in-a-pinch replication needs. It's like option C since RSYNC is just an
application. One feature of RSYNC is that it can avoid sending bits
which match what the receiver already has. I also like that it can be
told to not step on a newer copy (if source and target may each have
newer versions of some files).

I _have used RSYNC bi-directionally_. Given the specific options that I
most often use, my downside is the need to manually delete unwanted
files. (RSYNC has an option to delete files, but I never use it "just in
case" and it seems dangerous for bi-directional.)

I've _also seen option A used_ in rigorous off-site and heavily
exercised D/R. (We're talking greater than 300 miles.) Options A and C
both allow for any filesystems you might need or choose. (Not that
there's anything wrong with option B, FS based solutions.) Option A
often requires that you go deep with a specific storage vendor or service.

The option A work that I was involved in was for a former employer. Most
of the exercise was truly outstanding, excellent work. But procedures
historically included _applying updates via tape_ (after snap-shot of
the storage across the 300-mile link). I am just not a fan of tape
anymore for a variety of reasons. Was annoyed that the D/R exercise
would consistently burn many hours for all the tape work, but was not
the decision maker.

I'm a huge fan of _shared disk images_, even within only one data
center, so I strongly advocate option A for static content. (And most
systems on your SAN fabric probably grok ISO9660.)

-- R; <><


On 01/31/2017 11:04 PM, Angelo McComis wrote:
> I see a fair amount of this in various customer environments all the
> time. I've also been around BCP/Disaster Recovery, so I can speak
> about the choices, what they do, what they don't do, and so on. If you
> have a specific question, you're welcome to reach out. But, sharing
> experience, in general, I can share this off the top of my head...
>
> Multi-site replication comes in a few different varieties: 
>
> A) Storage-based (e.g. your SAN/NAS device is shipping block-by-block
> changes to a matching storage system at the other site, which is
> applying those changes/writes to a remote mirror)
> B) Host-based (e.g. your OS file system driver is taking IO writes for
> a given filesystem and replicating them to another host, which that
> other host is applying those writes to a remote filesystem)
> C) Application-based (e.g. some built-in feature to an application
> [not unlike mysql] is pushing bits over the network to another copy
> somewhere else, that is receiving those bits)
>
> In cases of A and B above, and sometimes, but not always C, the remote
> side is in a "read only" mode and not usable, since there's no
> mechanism to take writes on the far side and get them back to the
> original site.  In the case of application-based replication, there
> are some that handle bi-directional replication, so you're not stuck
> with a far site in read-only mode.
>
> In case of A above, you can ensure zero data loss if you have 1) short
> enough distance between the two sites, and 2) enabled synchronous
> replication.  In this use case, a write is written to the A side,
> replicated to the B side, and confirmation of the write is sent back
> to A before the A side considers the write to be complete and releases
> control back to the requesting application. Because of the extra hops,
> synchronous replication is limited to ~20 miles of distance because of
> latency. If zero data loss is not an absolute requirement,
> asynchronous replication is not distance limited, but as distance
> increases, the lag between a write to the near side and that write
> being committed to the far side increases as well.  
>
> In your specific mentioning of XtreemFS, it works like option B above,
> as it plugs in at the filesystem driver layer of the OS, and appears,
> from my reading of their site info, to operate asynchronously.
> GlusterFS is similar. It's asynchronous, and can work across distance
> to create global clusters.
>
> The art of applying a replication strategy is to first understand what
> the business or technical requirement is that must be met. What are
> some of the use cases?  If you're wondering what of the above I see
> most often? It's A and C. Examples: Frame-to-Frame storage replication
> done at the volume level, databases running log-shipping to remote
> sites for a DR copy, that sort of thing. 
>
> Angelo
>
>
>
>
>
> On Tue, Jan 31, 2017 at 9:18 PM, Christopher Cavello
> <cavello.1 at osu.edu <mailto:cavello.1 at osu.edu>> wrote:
>
>     Is anyone on this list willing to share his experience with file
>     replication across data centers? (glusterfs geo-replication, xtreemfs
>     http://www.xtreemfs.org/, etc.)
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.colug.net/pipermail/colug-432/attachments/20170201/1d815cdf/attachment.html