Discussion:
Following symlinks in globstar
Chris Down
2013-08-01 11:02:31 UTC
Permalink
As we're probably all aware, `globstar' follows symlinks when doing recursive
traversal. Is it possible to, at some future version, have an option that
enables/disables (I guess enables by default for backwards compatibility)
following symlinks? This can be quite irritating when trying to traverse procfs
filesystems especially.
Chris Down
2014-01-21 11:30:53 UTC
Permalink
Post by Chris Down
As we're probably all aware, `globstar' follows symlinks when doing
recursive traversal. Is it possible to, at some future version, have
an option that enables/disables (I guess enables by default for
backwards compatibility) following symlinks? This can be quite
irritating when trying to traverse procfs filesystems especially.
It's been nearly half a year since I sent this (as yet unreplied to)
message, and I still wonder what people's thoughts are on having some
path to better globstar modularity.

I still think the best path is to avoid breaking backwards
compatibility, and to provide a shell option to disable globstar
traversing symlinks.

Another option is to break backwards compatibility and do the (arguably)
more logically expected thing by having ** not traverse symlinks, and
*** traverse symlinks. That's what a few people I know propose, but I
think it is quite probably that the symlink-traversing properties of **
are used in production now.

I am not so worried about the method used to "fix" globstar -- whether
we keep backwards compatibility or not -- I am more concerned that we
have at least *some* way of disabling what many people view as
undesirable behaviour.
Chet Ramey
2014-01-21 15:19:10 UTC
Permalink
Post by Chris Down
Post by Chris Down
As we're probably all aware, `globstar' follows symlinks when doing
recursive traversal. Is it possible to, at some future version, have
an option that enables/disables (I guess enables by default for
backwards compatibility) following symlinks? This can be quite
irritating when trying to traverse procfs filesystems especially.
It's been nearly half a year since I sent this (as yet unreplied to)
message, and I still wonder what people's thoughts are on having some
path to better globstar modularity.
I still think the best path is to avoid breaking backwards
compatibility, and to provide a shell option to disable globstar
traversing symlinks.
Another option is to break backwards compatibility and do the (arguably)
more logically expected thing by having ** not traverse symlinks, and
*** traverse symlinks. That's what a few people I know propose, but I
think it is quite probably that the symlink-traversing properties of **
are used in production now.
I am not so worried about the method used to "fix" globstar -- whether
we keep backwards compatibility or not -- I am more concerned that we
have at least *some* way of disabling what many people view as
undesirable behaviour.
It may or may not be undesirable -- we have no data other than your
original message indicating that it is. There have been no proposals,
no code contributions, and nothing other than anecdotes to demonstrate
that "many people" view it as a problem.

I interpret this lack of discussion as an indication of priority. I do
not consider this a priority, and have not to this point, and I have only
looked at it briefly.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU ***@case.edu http://cnswww.cns.cwru.edu/~chet/
Stephane Chazelas
2014-01-25 23:11:29 UTC
Permalink
2014-01-21 10:19:10 -0500, Chet Ramey:
[...]
Post by Chet Ramey
Post by Chris Down
I am not so worried about the method used to "fix" globstar -- whether
we keep backwards compatibility or not -- I am more concerned that we
have at least *some* way of disabling what many people view as
undesirable behaviour.
It may or may not be undesirable -- we have no data other than your
original message indicating that it is. There have been no proposals,
no code contributions, and nothing other than anecdotes to demonstrate
that "many people" view it as a problem.
I interpret this lack of discussion as an indication of priority. I do
not consider this a priority, and have not to this point, and I have only
looked at it briefly.
[...]

I agree with Chris that the bash behaviour here is not
desirable. There's a number of articles on usenet or
unix.stackexchange.com (many by me sorry,
http://unix.stackexchange.com/search?q=bash+globstar+symlink)
that say that the bash behaviour is broken in that regard and
that globstar from other shells should be used instead.

That following symlink behaviour has plagged GNU grep for years
until it was fixed in 2.12 (breaking backward compatibility),
where -r now does not follow symlinks anymore (but -R still does).

The **/ feature comes from zsh (1990), was first copied by ksh93
in 2003 (with differences), by fish in 2005, bash in 2009 and
tcsh in 2010.

bash's behaviour in many respect is closer to ksh's than zsh's.

fish's is the only other implementation that follows symlinks.

zsh's, ksh's, tcsh's don't.

zsh and tcsh both support ***/* to follow symlinks.

Utilities that recurse (chmod -R, chown -R, find, cp -r...) do
not follow symlinks (some have options to allow it).

When they do, at least they try to detect loops, bash doesn't
(fish seems to have some sort of provision for it?)

IMO, it's a dangerous misfeature, prone to introducing security
vulnerabilities.

I beleive most people using globstar are unaware of the problem,
and changing the behaviour to not follow symlinks would a lot
more likely fix those scripts that use globstar than break them.

In over 15 years of using zsh, I've been using ** daily but I
can probably count on the fingers of my hands the times I've
used *** (certainly a lot less than the times I've been bitten
by grep -r following symlinks).

Now granted, */* does follow symlinks in every shell. But
there's no recursion there, so there are a few issues which it
doesn't have.

Where bash could improve on zsh here is provide a way to avoid
following symlinks in both **/* and */*/*.

Cheers,
Stephane
Chet Ramey
2014-02-04 14:23:21 UTC
Permalink
Post by Stephane Chazelas
[...]
Post by Chris Down
I am not so worried about the method used to "fix" globstar -- whether
we keep backwards compatibility or not -- I am more concerned that we
have at least *some* way of disabling what many people view as
undesirable behaviour.
[...]
Post by Stephane Chazelas
I agree with Chris that the bash behaviour here is not
desirable. There's a number of articles on usenet or
unix.stackexchange.com (many by me sorry,
http://unix.stackexchange.com/search?q=bash+globstar+symlink)
that say that the bash behaviour is broken in that regard and
that globstar from other shells should be used instead.
I put in a tentative change; we will see how it works.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU ***@case.edu http://cnswww.cns.cwru.edu/~chet/
Stephane Chazelas
2014-03-18 13:50:47 UTC
Permalink
Post by Chet Ramey
Post by Stephane Chazelas
[...]
Post by Chris Down
I am not so worried about the method used to "fix" globstar -- whether
we keep backwards compatibility or not -- I am more concerned that we
have at least *some* way of disabling what many people view as
undesirable behaviour.
[...]
Post by Stephane Chazelas
I agree with Chris that the bash behaviour here is not
desirable. There's a number of articles on usenet or
unix.stackexchange.com (many by me sorry,
http://unix.stackexchange.com/search?q=bash+globstar+symlink)
that say that the bash behaviour is broken in that regard and
that globstar from other shells should be used instead.
I put in a tentative change; we will see how it works.
[...]

That looks a lot better indeed. Thanks Chet.

regards,
Stephane

Loading...