Slony-I is a "master to multiple slaves" replication system supporting cascading (e.g. - a node can feed another node which feeds another node...) and failover.

The big picture for the development of Slony-I is that it is a master-slave replication system that includes all features and capabilities needed to replicate large databases to a reasonably limited number of slave systems.

Slony-I is a system designed for use at data centers and backup sites, where the normal mode of operation is that all nodes are available.

A fairly extensive "admin guide" comprising material in the CVS tree may be found here. There is also a local copy.

The original design document is available here.

See the "news" area for more details, including a copy of the release notes. This version fixes quite a number of issues found in early use of version 2.0.

See the "news" area for more details, including a copy of the release notes. This version fixes issues relating to FAILOVER.

Source RPMs (SRPMs) are available

See the "news" area for more details, including a copy of the release notes. This version fixes quite a number of issues found in early use of version 2.0.0.

See the "news" area for more details, including a copy of the release notes.

This is a major new release of Slony-I; it makes use of some features introduced in PostgreSQL 8.3, and hence is not compatible with versions older than 8.3.

This is considered a pretty good tradeoff, as various functionality would not be possible with earlier versions of PostgreSQL.

  • Internal catalogues are no longer "hacked with," so that you may, with the new version, use pg_dump against subscribers and be able to expect to have a complete and consistent dump.
  • Trigger handling is enormously cleaner.

Unfortunately, it needs to be noted that there is not, as of yet, an upgrade procedure to upgrade an installation of Slony-I 1.2.x to 2.0. At present, upgrading to 2.0 will essentially require dropping out replication and recreating it.

We hope to have an answer as to how to deal with this in the new year.

Follow Bug #69 for more details on this.

Version 1.2.15 is now available. See the "news" area for more details, including a copy of the release notes.

Slony-1 2.0.2 engine documentation
Slony-1 1.2.16 engine documentation
Slony-1 1.1.9 engine
Chris Browne 2008-09-12
Version slony1-2.0.2.tar.bz2 has been released.

Changes since 2.0.1 are as follows:

  • PostgreSQL 8.4 compatibility - src/backend/slony1_funcs.c needed to #include "catalog/pg_type.h"; that is no longer implicitly included by the SPI #include
  • Fix bug #71 - config dump was failing on string entries that were NULL
  • Fix bug #76 - REPAIR CONFIG could have collisions where, when we load a schema, one entry in sl_table already has an OID value that incorrectly matches a value that another update is trying to fix.

    This would cause REPAIR CONFIG to fail.

    Put in a query that updates the oids in sl_table and sl_sequence to randomly selected values from pg_class that couldn't match (since the values selected are from non-table/non-sequence objects)

  • Updated release notes to indicate that 2.0 only supports PostgreSQL 8.3+
  • Change to configure.ac to allow it to run on autoconf 2.63; there was something unsuitable about SLONREL_VERSION processing
  • Fixes to Makefile for admin guide
  • Various fixes to admin guide material
  • Added an extra test to Schema name test that validates that things can operate properly even if the DB lacks a "public" schema
  • Fixed bug in tools/start_slon.sh
  • Fixed pathname substitutions in a couple of altperl scripts
  • Add a test which validates that the cluster schema name is valid; pgAdminIII allowed setting up cluster names that slonik would refuse to connect to.
  • Revised regression tests to use SYNC/WAIT FOR EVENT instead of queries that try to coordinate sl_status information with rather less determinacy...
Chris Browne 2009-05-08
Version slony1-1.2.16.tar.bz2 has been released. Changes since 1.2.15:
  • Fix problems with failover:
    • Failover would fail (ERROR) where there is no path to backup node - change this to a WARNING so that failover can complete; admin will need to fix up paths later
    • Failover apparently never supported failing over to nodes that were not directly subscribing to the failed node Fixed some broken logic in failedNode()
    • Fixed a problem where slonik cannot perform a failover if multiple origins fail at once.
  • Fixed autoconf 2.63 incompatibility
  • Draw in latest documentation updates from 2.0 branch
Chris Browne 2009-05-08
The release candidate slony1-2.0.2-rc.tar.bz2 has been generated.
  • PostgreSQL 8.4 compatibility - src/backend/slony1_funcs.c needed to #include "catalog/pg_type.h"; that is no longer implicitly included by the SPI #include
  • Fix bug #71 - config dump was failing on string entries that were NULL
  • Fix bug #76 - REPAIR CONFIG could have collisions where, when we load a schema, one entry in sl_table already has an OID value that incorrectly matches a value that another update is trying to fix.

    This would cause REPAIR CONFIG to fail.

    Put in a query that updates the oids in sl_table and sl_sequence to randomly selected values from pg_class that couldn't match (since the values selected are from non-table/non-sequence objects)

  • Updated release notes to indicate that 2.0 only supports PostgreSQL 8.3+
  • Change to configure.ac to allow it to run on autoconf 2.63; there was something unsuitable about SLONREL_VERSION processing
  • Fixes to Makefile for admin guide
  • Various fixes to admin guide material
  • Added an extra test to Schema name test that validates that things can operate properly even if the DB lacks a "public" schema
  • Fixed bug in tools/start_slon.sh
  • Fixed pathname substitutions in a couple of altperl scripts
  • Add a test which validates that the cluster schema name is valid; pgAdminIII allowed setting up cluster names that slonik would refuse to connect to.
  • Revised regression tests to use SYNC/WAIT FOR EVENT instead of queries that try to coordinate sl_status information with rather less determinacy...
Chris Browne 2009-04-30
The release candidate slony1-1.2.16-rc.tar.bz2 has been generated. Changes:
  • Fix problems with failover:
    • Failover would fail (ERROR) where there is no path to backup node - change this to a WARNING so that failover can complete; admin will need to fix up paths later
    • Failover apparently never supported failing over to nodes that were not directly subscribing to the failed node Fixed some broken logic in failedNode()
    • Fixed a problem where slonik cannot perform a failover if multiple origins fail at once.
  • Fixed autoconf 2.63 incompatibility
  • Draw in latest documentation updates from 2.0 branch
Chris Browne 2009-04-30
Slony1-ctl is a set of shell scripts aiming at simplifying everyday administration of a Slony1 replication.

Licence: BSD

Principle:

Once replication nodes and relations between nodes have been defined, one can easily administrate the replication by calling one of the scripts, without need to write slonik scripts anymore. Features:

  • Initialising a replication
  • Adding/dropping nodes
  • Adding a set
  • Possibility to define cross-replication
  • adding objects
  • switchover/failover
  • Start/stop replication daemons
  • Executing DDL scripts
Chris Browne 2009-04-13

This release addresses a number of issues found in 2.0.0

  • Bug #69 - vactables type was being referenced before UPGRADE FUNCTIONS could actually have created it, when upgrading from an earlier version.

    This would cause UPGRADE FUNCTIONS to fail.

  • Bug #64 - use of syslog levels had some inappropriate break statements so that some logging would get lost.

    If you had multiple clusters where one cluster name was a substring of the other, slon-tools.pm could find both when looking for slon processes relevant to the shorter cluster name. Added a space to the relevant bit of slon-tools.pm

  • Failover had an insert into sl_event where # of columns provided didn't match those expected. This resulted from the change of snapshot handling which used to involve 3 columns where in 8.3+, there is now a snapshot type.
  • Fix syntax error in tools/altperl/slon_watchdog.pl
  • Add cleanup_interval example to sample slon.conf file
  • Add cleanup_interval configuration configuration to regression tests
  • Add documentation for complex failover scenario
  • Revise installation notes to indicate that xxid.so no longer exists in Slony-I 2.0+
  • Fix documentation typos
  • Fixed error with cleanup code which was not purging old data in sl_log_1/sl_log_2
  • Fix altperl/slontools errors
  • Added in preliminaries for upgrade from 1.2 to 2.0 - discussion of what the tool needs to do
  • Bug #63 - Array overrun in logtrigger()
Chris Browne 2009-02-26

Unfortunately, it needs to be noted that there is not, as of yet, an upgrade procedure to upgrade an installation of Slony-I 1.2.x to 2.0. At present, upgrading to 2.0 will essentially require dropping out replication and recreating it.

We hope to have an answer as to how to deal with this in the new year.

Chris Browne 2008-12-19
Differences from 1.2 stream
  • Removal of TABLE ADD KEY
  • It drops all support for databases prior to Postgres version 8.3.

    This is required because we now make use of new functionality in Postgres, namely the trigger and rule support for session replication role. As of now, every node (origin/subscriber/mixed) can be dumped with pg_dump and result in a consistent snapshot of the database.

  • Still need alterTableRestore() for the upgrade from 1.2.x to 2.0. upgradeSchema() will restore the system catalog to a consistent state and define+configure the new versions of the log and deny_access triggers.
  • Fix EXECUTE SCRIPT so that it records the ev_seqno for WAIT FOR EVENT and make sure all DDL is executed in session_replication_role "local" on the origin as well as all subscribers. This will cause the slony triggers to ignore all DML statements while user triggers follow the regular configuration options for ENABLE [REPLICA/ALWAYS] or DISABLE.
  • Let the logshipping files also switch to session_replication_role = "replica" or "local" (for DDL).
  • Sequence tracking becomes enormously less expensive; rather than polling *ALL* sequence values for each and every SYNC, the slon stores the last value, and only records entries in sl_seqlog when the value changes from that last value. If most sequences are relatively inactive, they won't require entries in sl_seqlog very often.
  • Change to tools/slony1_dump.sh (used to generate log shipping dump); change quoting of "\\\backslashes\\\" to get rid of warning
  • Cleanup thread revised to push most of the logic to evaluate which tables are to be vacuumed into a pair of stored functions.

    This fairly massively simplifies the C code.

  • Revised logging levels so that most of the interesting messages are spit out at SLON_CONFIG and SLON_INFO levels. This can allow users to drop out the higher DEBUG levels and still have useful logs.
  • Changed log selection query to be less affected by long running transaction. This should help, in particular, the scenario where it takes a very long time to subscribe to a set. In that situation, we have had the problem where applying the later SYNCs gets extremely costly as the query selecting logs wound up forced into a Seq Scan rather than an index scan.
  • Removed all support for STORE/DROP TRIGGER commands. Users should use the ALTER TABLE [ENABLE|DISABLE] TRIGGER functionality available directly in Postgres from now on.
  • Improve Wiki page generation script so that it has an option to add in a set of [[Category:Foo]] tags to allow automated categorization.
  • Documented how to fix tables that presently use Slony-I-generated primary key candidates generated by TABLE ADD KEY
  • Add some specific timestamps during the 2007 "DST rule change ambiguous time" (e.g. - during the period which, under former rules, was not DST, but which now is, due to the recent rule change).

    Bill Moran ran into some problems with such dates; varying PostgreSQL versions returned somewhat varying results. This wasn't a Slony-I problem; the data was indeed being replicated correctly.

  • Made configure a bit smarter about automatically locating docbook2man-spec.pl on Debian, Fedora, BSD.
  • Tests now generate |pipe|delimited|output| indicating a number of attributes of each test, including system/platform information, versions, and whether or not the test succeeded or failed.
  • Revised functions that generate listen paths
  • tools/configure-replication.sh script permits specifying a destination path for generated config files. This enables using it within automated processes, and makes it possible to use it to generate Slonik scripts for tests in the "test bed," which has the further merit of making tools/configure-replication.sh a regularly-regression-tested tool.
  • Fix to bug #15 - where long cluster name (>40 chars) leads to things breaking when an index name is created that contains the cluster name.

    Warn upon creating a long cluster name.

    Give a useful exception that explains the cause rather than merely watching index creation fail.

    Bug 15

  • Fix for bug #19 - added a script to help the administrator search for any triggers on the database that is the source for a schema that is to be used to initialize a log shipping node.

    The problem is that some/most/sometimes all triggers and rules are likely to need to be dropped from the log shipping node lest they interfere with replication.

  • Elimination of custom "xxid" functions

    PostgreSQL 8.3 introduces a set of "txid" functions and a "txid_snapshot" type, which eliminates the need for Slony-I to have its own C functions for doing XID comparisons.

    Note that this affects the structure of sl_event, and leads to some changes in the coding of the regression tests.

    This eliminates the src/xxid directory and contents

  • All of the interesting cleanup work is now done in the stored function, cleanupEvent(interval, boolean).

    Interesting side-effect: You can now induce a cleanup manually, which will be useful for testing.

  • cleanupEvent now has two parameters, passed in from slon config parameters:

    interval - cleanup_interval (default '10 minutes')

    This controls how quickly old events are trimmed out. It used to be a hard-coded value.

    Old events are trimmed out once the confirmations are aged by (cleanup_interval).

    This then controls when the data in sl_log_1/sl_log_2 can be dropped.

    Data in *those* tables is deleted when it is older than the earliest XID still captured in sl_event.

    boolean - cleanup_deletelogs (default 'false')

    This controls whether or not we DELETE data from sl_log_1/sl_log_2

    By default, we now NEVER delete data from the log tables; we instead use TRUNCATE.

  • We now consider initiating a log switch every time cleanupEvent() runs.

    If the call to logswitch_finish() indicates that there was no log switch in progress, we initiate one.

    This means that log switches will be initiated almost as often as possible. That's a policy well worth debating :-).

  • logswitch_finish() changes a fair bit...

    It uses the same logic as in cleanupEvent() to determine if there are any *relevant* tuples left in sl_log_[whatever], rather than (potentially) scanning the table to see if there are any undeleted tuples left.

  • At slon startup time, it logs (at SLON_CONFIG level) all of the parameter values. Per Bugzilla entry #21.
  • New slonik "CLONE PREPARE" and "CLONE FINISH" command to assist in creating duplicate nodes based on taking a copy of some existing subscriber node.
  • We no longer use LISTEN/NOTIFY for events and confirmations, which eliminates the usage that has caused pg_listener bloat. We instead poll against the event table.
  • Various instances where slonik would use a default node ID of 1 have been changed to remove this.

    Slonik scripts may need to be changed to indicate an EVENT NODE (or similar) after migration to v2.0 as a result.

    The slonik commands involved:

    • STORE NODE - EVENT NODE
    • DROP NODE - EVENT NODE
    • WAIT FOR EVENT - WAIT ON
    • FAILOVER - BACKUP NODE
    • EXECUTE SCRIPT - EVENT NODE
  • Fixed a problem where ACCEPT_SET would wait for the corresponding MOVE_SET or FAILOVER_SET to arrive while holding an exclusive lock on sl_config_lock, preventing the other remote worker to process that event.
  • Bug #54 - quite a few Bash-isms in various scripts have been addressed so as to make the shell scripts more portable.
  • Bug #18 - the function parameter for the logtrigger functions no longer requires any trailing v's

    Add a test to "test1" to make sure this logic gets exercised.

  • Created "start_slon.sh", an rc.d-style script for starting, stopping, and checking status of slon processes.

    Integrated this into the regression tests, replacing previous logic for starting/stopping slons, so that this script can be considered carefully tested

  • Bug #46 - incompatibility with PostgreSQL 8.4 addressed
  • Use dollar quoting in stored functions
  • Additional logging of the time spent running queries, broken out on a by-database basis
  • Fixes to documentation of WAIT FOR EVENT
  • Fix to bug #63 - cleanup thread had an imperative SELECT that needed to become part of an IF statement
  • Enhancement - bug #61 - logshipper process should rescan the queue when it empties
  • Note about "duct tape" tests: There are many of these tests that reside in src/ducttape that reference features removed in v2.0.

    We will eventually be replacing these with a more proper "test suite" so we're not remedying all the ducttape tests.

Chris Browne 2008-11-24
Now available is a second release candidate of version 2.0 of Slony-I, fixing a number of issues found since the first release candidate.
Differences from 2.0.0 RC1
  • Bug #54 - fixed various bash-isms
  • Bug #18 - function parameter for logtrigger no longer requires trailing "v"'s
  • Created start_slon.sh, an rc.d-style script to start, stop, and check status of a slon process, integrating its usage into regression tests
  • Bug #46 - incompatibility with PostgreSQL 8.4
  • Added an extended example of upgrading from one version of PostgreSQL to another
  • Use dollar quoting in stored functions
  • Added logging code that lists time spent running queries against different nodes to help when analyzing performance issues
  • A little more fixing-up of logging levels
  • Test suite uses non-zero lag_interval
Chris Browne 2008-09-24
Version 1.2.15 has been released. Changes include:
  • Fix to STORE TRIGGER

    - store trigger was running against all nodes upon subscription

    Bug #56

  • Portability changes to some tools/ scripts, fixing some "bash-isms"
  • Fix to switch statement in slonik.c; unknown how it broke
  • Fix Bug #52 - memory leak
Chris Browne 2008-09-12