DO NOT REPLY [Bug 7249] New: Add an option to use O_NOATIME

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

DO NOT REPLY [Bug 7249] New: Add an option to use O_NOATIME

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249

           Summary: Add an option to use O_NOATIME
           Product: rsync
           Version: 3.1.0
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: enhancement
          Priority: P3
         Component: core
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]
         QAContact: [hidden email]


Some systems (at least Linux since 2.6.8) have the O_NOATIME open flag that
avoids altering the atime of files opened for reading. An option to enable this
flag is supported would be useful for backup systems based on rsync, as it
would avoid interfering with tools that relies on atime (such as new mail
detection, whatever may be thought about the correctness of the method) and
reduce disc activity on the source filesystem.

We use a patched rsync that always sets O_ATIME with rsnapshot since quite some
time and we have not noticed any drawback.


--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

DO NOT REPLY [Bug 7249] Add an option to use O_NOATIME

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249





------- Comment #1 from [hidden email]  2010-03-14 17:09 CST -------
Created an attachment (id=5497)
 --> (https://bugzilla.samba.org/attachment.cgi?id=5497&action=view)
adds a --noatime option


--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

DO NOT REPLY [Bug 7249] Add an option to use O_NOATIME

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249


[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




------- Comment #2 from [hidden email]  2010-08-22 06:15 CST -------
this looks very useful . i vote for it.

seems there is already an atimes.diff in [rsync-patches.git] which has a
similar purpose !?


--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

DO NOT REPLY [Bug 7249] Add an option to use O_NOATIME

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249





------- Comment #3 from [hidden email]  2010-08-22 07:49 CST -------
They have a similar purpose, but they do not work the same way at all.

My patch (comment #2) politely asks the kernel not to update the atime of
opened files. It only works if the O_NOATIME flags exists on that particular
platform but has no other drawbacks.

AFAIK, the patch on rsync-patches.git explicitly puts back the atime to what it
was. It has the drawback of updating the ctime instead.


--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 7249] Add an option to use O_NOATIME

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249

Martin von Gagern <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #4 from Martin von Gagern <[hidden email]> 2011-12-10 19:53:32 UTC ---
+1 vote.

Should someone set the review flag to '?' to get Nicolas' patch reviewed? Who?

rsync3.txt in the source tree has this to say about atimes:

"Desirable features:
 [...]
 - Propagate atimes and do not modify them.  This is very ugly on
   Unix.  It might be better to try to add O_NOATIME to kernels, and
   call that."

So apparently someone with commit privileges (Martin Pool to be precise)
already has decided that such a feature would be desirable (back in 2001). Does
this add weight to this request and patch?

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 7249] Add an option to use O_NOATIME

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249

Martin von Gagern <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #9278|                            |review?
              Flags|                            |

--- Comment #5 from Martin von Gagern <[hidden email]> 2013-10-13 09:21:06 UTC ---
Created attachment 9278
  --> https://bugzilla.samba.org/attachment.cgi?id=9278
--noatime option for 3.1.0

I rebased Nicholas' patch to 3.1.0, with modifications as described below.

One thing I'm not sure is whether the --noatime flag should automatically be
forwarded to the server. In a general backup scenario, the client would wish to
preserve atimes, but on the backup server I see no reason to do so. Therefore I
made the forwarding part a comment. Users can still explicitely include the
--noatime in the list of server arguments if they whish this behaviour on both
ends.

I've been using this form on 3.0.9 for quite some time, and would like to see
this included officially one day soon. I hope setting it to “review=?” the way
I did is the way to go for that.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 7249] Add an option to use O_NOATIME

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249

--- Comment #6 from [hidden email] ---
+1 from here as well.

I understand the arguments about the limited usefulness of the atime field in
general, and I agree with much of it, but to me there are more use cases than
what's mentioned in what I've read so far - such as providing helpful clues for
certain kinds of "forensic" debugging.  atime has been around for a very long
time (every *NIX variant I've worked on, going back to the 1980's), so even
though its utility may be a bit "specialized", there are clearly people who are
surprised to see it ignored by a tool like rsync.  IMO back-up/replication/sync
tools should strive to record or replicate the "originals" as faithfully as
possible[*], with minimal impact on the contents or meta-data of files being
backed up or replicated.

That is to say, one should ideally be able to use rsync to do things like
moving a directory tree to a new place (file system, machine), or
copying/sync'ing it to a back-up location, and later restoring the originals,
without the user of those files being able to discern any differences in them.

I built rsync with Nicolas George's O_NOATIME patch on the latest Git, and IMO
this change brings rsync one step closer to that ideal (on Linux at least).  It
not only avoids clobbering the atime values for the original files as they're
being read during a first copy operation, but also when they're being
checksummed (on both sides) if one uses the "-c" option in a later re-sync.

Martin von Gagern: That's the main reason that I think "--noatime" should be
sent to the remote side.  I agree that in some/many cases, rsync users would
not care about atimes being preserved in the copy.  For backups, supposing
atimes had been copied to the backup machine[*] (perhaps not part of the
scenario you had in mind), they would need to be preserved during re-syncs in
order to be able to available when sync'd back to the original place during a
restore.

One problem with unconditionally sending a new option to the remote side is
that if it isn't supported on the remote side, it fails, whereas I would still
see benefit (for some use cases) in using O_NOATIME on the local side only in
that case.  Not sure how easy it would be to handle that situation.

Aside from that, I would really like to see this enhancement be integrated as
soon as possible.  Since it's an option, and doesn't affect the protocol etc,
it doesn't seem as though it would have any negative impact when it isn't being
used.  Seems like a win-win. :)  Could it be integrated before the next
release?


[*] Which brings us to the --atime patch -- but that's another topic.

--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 7249] Add an option to use O_NOATIME

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249

--- Comment #7 from [hidden email] ---
+1
I'm currently using a locally patched version, because I'm using rsync to
mirror my whole data daily, plus using a deduplicating backup tool weekly that
stores atime (so if rsync sets atime, the metadata are never deduplicated).
It is logical for a backup tool to avoid setting atime, and logical that it is
an option for rsync that may or may not be used as a backup tool.

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 7249] Add an option to use O_NOATIME

Samba - rsync mailing list
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=7249

--- Comment #8 from Michal Ruprich <[hidden email]> ---
Is there any update on this feature? Is there a plan to merge the patch into
master branch of rsync?

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html