Upgrading to GlusterFS 3.3.0
Now that GlusterFS 3.3.0 is out, here is a quick primer on upgrading from earlier installed versions of GlusterFS. This howto covers upgrades from 3.1.x and 3.2.x versions of GlusterFS.
1) GlusterFS 3.3.0 is not compatible with any earlier released versions. Please make sure that you schedule a downtime before you upgrade.
2) Stop all glusterd, glusterfs and glusterfsd processes running in all your servers and clients.
3) Take a backup of glusterd’s working directory on all servers – usually glusterd’s working directory is /etc/glusterd.
4) Install GlusterFS 3.3.0 on all your servers and clients. With 3.3.0, the default working directory has been changed to /var/lib/glusterd. RPM and source installations move all content under /etc/glusterd to /var/lib/glusterd. On all servers, ensure that the directory structure remains consistent with the backup obtained in 3). New files are created as part of the upgrade process and hence you may see some additional files. However, ensure all files backed up in 3) are present in /var/lib/glusterd.
5) If you have installed from RPM, goto 6). Else, start glusterd in upgrade mode. glusterd terminates after it performs the necessary steps for upgrade. Re-start glusterd normally after this termination. Essentially this process boils down to:
a) killall glusterd
b) glusterd --xlator-option *.upgrade=on -N
This will re-generate volume files with the new ‘index’ translator which is needed for features like pro-active self heal in 3.3.0.
c) Start glusterd
Ensure that you repeat a), b) and c) on all servers.
6) All your gluster services on the servers should be back online now. Mount your native clients with GlusterFS 3.3.0 to access your volumes.
Enjoy your shiny new 3.3.0 and report your findings in #gluster on Freenode, bugzilla and c.g.o
Please note that this may not work for all installations & upgrades. If you notice anything amiss and would like to see it covered here, please leave a comment and I will try to incorporate your findings.
10 Comments »
Leave a Reply
-
Recent
-
Links
-
Archives
- March 2013 (1)
- May 2012 (1)
- March 2011 (1)
- January 2011 (1)
-
Categories
-
RSS
Entries RSS
Comments RSS
Will there be an “hot” upgrade path / protocol compatibility in the future? We are using gluster for HA and bringing down the entire infrastructure for an upgrade defeats the whole purpose of it.
Yes, protocol compatibility is something we have tried to be conscious about with this release. Going forward, we will definitely try to retain backward compatibility so that an online rolling upgrade is possible.
Vijay wrote, “RPM and source installations move all content under /etc/glusterd to /var/lib/glusterd.” Do you mean to say that configuration files — which, unlike some files glusterd 2.2.x stores in /etc/glusterd (e.g., PID files created at runtime), should be in /etc — will instead be stored in a subdirectory of /var/lib?
No. /etc/glusterfs has (somewhat) user configurable files (though I don’t know anybody that’s altered them since 3.0).
/etc/glusterd had files that were created and managed by glusterd, the management interface. These are now correctly placed under the runtime state directory of /var/lib/glusterd. They are not intended to be directly configured by the administrator.
At item 5.b above the xlator-option should use two hifens like — and not one –
Trying again use two adjacent hifens like – - without the space between them.
Hi, I tried to upgrade from 3.2.1 to 3.3.0. It looked fine at first sight. But I found that adding new 3.3.0 nodes to the upgraded cluster fails. It’s probably because 3.2.1′s volume config lacks “username/password” authentication entries in “info” and “…vol”. My workaround is:
# gluster vol stop
# service glusterd stop
Modify above files by hand (adding username/password auth entries.)
# service glusterd start
# gluster vol reset
# gluster vol start
Now, I can add new 3.3.0 nodes with:
# gluster peer probe
Do you think this works well for all cases?
Hi, I created the whole process as stated above from 3.2.6 to 3.3.1 on #16 nodes (Centos 6.2 64bit, distributed 2 replicas, single volume)
I do not know why, but I had to restart some of the servers to work properly the whole system. The client couldnot connect to some servers, but the servers (peers) saw eac other according to peers status. The existing migrated volume was created in two different steps in the past (i.e. half of the servers joined 3 months later), and I had to restart those servers that joined in the second step. But after restarting, everything worked. The downtime was less than an hour. It was usefull seeing the client log of the affected volume for warnings.
I hope it helps others.
The upgrade step isn’t working for me (going from 3.2.5 to 3.3.1), this command just seems to be wrong:
# glusterd –xlator-option *.upgrade=on -N
Usage: glusterd [OPTION...] –volfile-server=SERVER [MOUNT-POINT]
or: glusterd [OPTION...] –volfile=VOLFILE [MOUNT-POINT]
Try `glusterd –help’ or `glusterd –usage’ for more information.
I tried adding a ‘–volfile=/etc/glusterd/vols/shared/shared-fuse.vol’ option but it didn’t make any difference. It’s not clear which volfile I’m supposed to use as there are several with similar names – and I only have one 4G volume with 2 nodes!
I have a /etc/glusterfs folder as well as /etc/glusterd – Gluster 3.2.5 seems to be updating things in both – is that expected?
> # glusterd –xlator-option *.upgrade=on -N
I am sorry for the inconvenience caused. This was a result of wordpress automatically converting double hypens to a single dash. I have updated the post to workaround this auto conversion.