Worth mentioning in documentation

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Worth mentioning in documentation

Juanma
Hello.

In section “3.2.4.2 Conditional Constructs” of Bash docs (both in ‘man’ and in the web), at the end, there is an explanation on combining expressions, starting with the use of brackets.

There it would be worth to mention that:
1. you must escape the brackets, also inside script files
2. you have to leave blank/s between the brackets and the expression itself

Point 2 could be inferred merely from the way it's written in the docs., but I don't think it's clear that brackets should be escaped. It took me quite a while and a lot of fumbling to realize that. The rest of operators in this section, like ! or && don't need to be quoted; not even !, which is suspicious of having a different meaning for the shell. I don't know what makes brackets special in this sense, but I guess it should be mentioned.
--
Juanma Menéndez

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Greg Wooledge
On Thu, Aug 06, 2015 at 04:13:30PM +0200, Juanma wrote:
> In section ???3.2.4.2 Conditional Constructs??? of Bash docs (both in ???man??? and in the web), at the end, there is an explanation on combining expressions, starting with the use of brackets.
>
> There it would be worth to mention that:
> 1. you must escape the brackets, also inside script files
> 2. you have to leave blank/s between the brackets and the expression itself

Section 3.2.4.2 of http://www.gnu.org/software/bash/manual/bash.html
is inside 3.2.4 Compound Commands, which is inside 3.2 Shell Commands.
I believe you are talking about the section that discusses the [[ ... ]]
command.

Point 1: I truly have no idea what you mean.  What brackets are you
escaping, and how, and why?

Point 2: Spaces.

Chet chose to omit the phrase "The syntax of ___ is:" from the last few
commands in this section, but you should certainly treat:

[[ ... ]]

     [[ expression ]]

as if it were written:

[[ ... ]]

  The syntax of the [[ ... ]] command is:

     [[ expression ]]

If you treat [[ expression ]] as a formal syntax specifier like those of
"if" and "case", just a few paragraphs up, then the spaces already in
it should be considered as documented.

The manual doesn't explicitly point out that you need spaces after "if"
or "case".  (It fails to mention that you can *omit* the spaces in the
((...)) command, but that's a special exception.)

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Greg Wooledge
On Fri, Aug 07, 2015 at 10:00:53AM +0200, Juanma wrote:

> El Thu 6 of Aug, Greg Wooledge profirió estas palabras:
> > I believe you are talking about the section that discusses the [[ ... ]]
> > command.
>
> Yes, you are right. And I mean, concretely, the last part:
> | Expressions may be combined using the following operators, listed in decreasing order of precedence:
> | ( expression )
> | Returns the value of expression. This may be used to override the normal precedence of operators.
> | ...
>
> > Point 1: I truly have no idea what you mean.  What brackets are you
> > escaping, and how, and why?
>
> Those brackets I cited above: ( expression )

In the US we call those "parentheses", and we reserve the word "brackets"
(or "square brackets") for [ ].  I realize that the UK uses different
terminology.  Hence, the word is ambiguous and you should always type
the actual characters you mean.

> But I have to apologize because I have just realized that, while still being a shell issue, it isn't a [[ issue: it's an issue with [.
>
>  [ 'a' == "b" -o 1 -lt 2 -a  1 -gt 0 ]  => return 1
>
>  [ ( 'a' == "b" -o 1 -lt 2 ) -a  1 -gt 0 ]
> bash: syntax error near unexpected token `'a''
>
>  [ \( 'a' == "b" -o 1 -lt 2 \) -a  1 -gt 0 ]  => return 0
>
> That's the escaping I meant.

Yes, the [ and [[ commands have totally different syntax.  This is
confusing and unfortunate.  You just have to live with it.

[[ is a "shell keyword" which means that it has special parsing rules
that don't apply to ordinary commands.  Among other things, you can
use shell metacharacters like ( and && inside [[ without escaping them.

[ is an ordinary command (a "shell builtin"), so it has no special
parsing rules.  Arguments to [ have to be escaped or quoted, exactly
the same way you would have to escape or quote them if you were
passing them to expr(1) or find(1).

> > Point 2: Spaces.
> [...]
> > The manual doesn't explicitly point out that you need spaces after "if"
> > or "case".  (It fails to mention that you can *omit* the spaces in the
> > ((...)) command, but that's a special exception.)
>
> The thing with brackets is that they are operators (if I'm not wrong), but they need special treatment, compared to other operators: first, because they need escaping; second, because they need spaces around them, as if they were commands or operands:
>
>  [ \('a' == "b" -o 1 -lt 2\) -a  1 -gt 0 ]
> bash: [: 2): integer expression expected

P.S. == is not required to be supported by the [ command.  You should use =
instead.  Supporting == in [ is a bash extension.

> However, it seems like the quotes around a prevent the ambiguity in the case of the opening bracket:  \('a'  That causes no error.
>
> Am I making any point now?

You were reading the section on [[ and assuming that it applies to [.
That's a huge mistake.

(It also didn't help that you talked about escaping "brackets" when
the command name in question is two brackets.  But you apparently meant
inner ( ) arguments within the command.)

Many bash users recommend ONLY using [[ and abandoning [ altogether.
(I actually disagree, and I often prefer using the portable syntax
when it can do what's needed.  But you'll have to decide for yourself.)

If you choose to use [ then you should use it in a portable way
(otherwise, there is no point).  And the first thing you need to know
about using [ portably is that you CANNOT use -a or -o ("and" or "or")
operators in it.  Don't even try!  Once you get that out of your head,
then you don't need parentheses ("UK brackets") either.

POSIX says that the behavior of [ depends on how many arguments you
pass to it.  It has very limited capability.

Here's the documentation:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html

Basically, almost any expression involving any combination of tests is
unspecified.  You just can't combine them in a single test or [ command.
What you should do instead is use multiple test or [ commands.

if test "$a" = foo && test -s "$outputfile" ; then
...
fi

if [ ! -d "$dir1" ] || [ ! -d "$dir2" ] ; then
...
fi

That is how you write compound conditional expressions using portable
POSIX sh syntax (with the [ or test commands).

If you choose to use [[ (a bash extension, also found in ksh) then
the rules are entirely different.  Within [[ you can join compound
expressions using && and ||.

if [[ ! -d $dir1 || ! -d $dir2 ]] ; then
...
fi

Do not use -a or -o.  They might as well not EXIST.  Just forget them.
You can't use them in [ and you can't use them in test and you can't
use them in [[.

Use one of the syntaxes I've shown here.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Eduardo A. Bustamante López
> In the US we call those "parentheses", and we reserve the word "brackets"
> (or "square brackets") for [ ].  I realize that the UK uses different
> terminology.  Hence, the word is ambiguous and you should always type
> the actual characters you mean.
These are also "paréntesis" in Spanish, so it's not a translation issue.

--
Eduardo Bustamante
https://dualbus.me/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Juanma
In reply to this post by Greg Wooledge
El Fri 7 of Aug, Greg Wooledge profirió estas palabras:
> > Those brackets I cited above: ( expression )
> In the US we call those "parentheses", and we reserve the word "brackets"
> (or "square brackets") for [ ].  I realize that the UK uses different
> terminology.  Hence, the word is ambiguous and you should always type
> the actual characters you mean.

Lesson learned.
I didn't mean to enforce UK wording; I just forgot there was a more "natural" word for me (as Eduardo noted).

[...]
> [ is an ordinary command (a "shell builtin")

Here is another point I find confusing: I thought a "shell builtin" didn't have a separate binary executable file, like 'cd' (which cd => fail), but some of them do have such form (which [ => /usr/bin/[ ; which pwd => /bin/pwd). I also fail to see how 'test' modifies the state of the shell itself (like 'cd' does), or why it is "impossible or inconvenient to obtain [its functionality] with separate utilities".

But that's another story.

> You were reading the section on [[ and assuming that it applies to [.
> That's a huge mistake.
[...]
> Do not use -a or -o.  They might as well not EXIST.  Just forget them.
> You can't use them in [ and you can't use them in test and you can't
> use them in [[.
>
> Use one of the syntaxes I've shown here.

Huge thanks. It was a good explanation.
--
Juanma Menéndez

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Greg Wooledge
On Mon, Aug 10, 2015 at 10:18:52AM +0200, Juanma wrote:
> > [ is an ordinary command (a "shell builtin")
>
> Here is another point I find confusing: I thought a "shell builtin" didn't
> have a separate binary executable file, like 'cd' (which cd => fail), but
> some of them do have such form (which [ => /usr/bin/[ ; which pwd =>
> /bin/pwd). I also fail to see how 'test' modifies the state of the shell
> itself (like 'cd' does), or why it is "impossible or inconvenient to obtain
> [its functionality] with separate utilities".

Don't use which(1).  Which is an external program, so it has no knowledge
of the shell's builtins, aliases, functions and keywords.  Instead, use
type.

imadev:~$ type cd
cd is a shell builtin
imadev:~$ type [[
[[ is a shell keyword
imadev:~$ type -a test
test is a shell builtin
test is /usr/bin/test
test is /bin/test

Bash implements test as a builtin not because it's necessary, but simply
for efficiency.  Forking a whole process to test whether two strings are
equal would be horribly wasteful.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Eric Blake-3
In reply to this post by Juanma
On 08/10/2015 02:18 AM, Juanma wrote:

> Here is another point I find confusing: I thought a "shell builtin" didn't have a separate binary executable file, like 'cd' (which cd => fail),

Actually, POSIX requires that there be a separate 'cd' binary, although
it does not have to behave the same as the shell builtin.  (About all an
exec'able cd can do is tell you by exit status whether the builtin cd
would succeed or fail; or be used for its CDPATH side-effect of printing
a directory name).

GNU/Linux systems tend to ignore the POSIX requirement of exec'able
counterparts, although here is how Solaris effectively does it:

$ cat /bin/cd
#!/bin/sh
exec $(basename $0) "$@"
$

and hard-linking that 2-liner to all of the shell builtins where POSIX
requires to have a non-builtin counterpart.

See also http://austingroupbugs.net/view.php?id=705

It is only the special builtins (such as 'exit') where POSIX does not
require an exec'able counterpart.

--
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


signature.asc (617 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Bob Proulx
In reply to this post by Greg Wooledge
Greg Wooledge wrote:

> Juanma wrote:
> > > [ is an ordinary command (a "shell builtin")
> >
> > Here is another point I find confusing: I thought a "shell builtin" didn't
> > have a separate binary executable file, like 'cd' (which cd => fail), but
> > some of them do have such form (which [ => /usr/bin/[ ; which pwd =>
> > /bin/pwd). I also fail to see how 'test' modifies the state of the shell
> > itself (like 'cd' does), or why it is "impossible or inconvenient to obtain
> > [its functionality] with separate utilities".
>
> Don't use which(1).  Which is an external program, so it has no knowledge
> of the shell's builtins, aliases, functions and keywords.  Instead, use
> type.

Another problem with 'which' is that it was originally designed and
written for csh users specifically.  It was a csh script and would
source the users ~/.cshrc file so as to acquire their aliases from
there so as to be able to report user aliases.  I think it is still
that way on HP-UX (and probably others) for example.

However that obviously won't work well for bash, ksh, zsh, and other
shell users.  And IMNHO csh is a terrible shell regardless of the tcsh
users using it who think otherwise.  Therefore some distributions such
as Debian have rewritten 'which' as a /bin/sh script meaning that
'which' behaves differently on different systems.  It is non-portable.

> imadev:~$ type cd
> cd is a shell builtin
> imadev:~$ type [[
> [[ is a shell keyword
> imadev:~$ type -a test
> test is a shell builtin
> test is /usr/bin/test
> test is /bin/test
>
> Bash implements test as a builtin not because it's necessary, but simply
> for efficiency.  Forking a whole process to test whether two strings are
> equal would be horribly wasteful.

An important mental concept is that test and [ must *behave* the same
as if they were an external program.  They are internal builtins for
efficiency now but everything behaves the same if they are external
programs.  This is why quoting as if they were external programs is
required.  On the other hand [[ has always been a builtin and
therefore the shell can avoid one layer of quoting and does.

Bob

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Juanma
El Mon 10 of Aug, Bob Proulx profirió estas palabras:
> [...] This is why quoting as if they were external programs is
> required.  On the other hand [[ has always been a builtin and
> therefore the shell can avoid one layer of quoting and does.

That's a good point to make. Thanks for clarifying.
--
Juanma Menéndez
Tlf: 1118
Skype: juanma_bellon

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Worth mentioning in documentation

Pádraig Brady
In reply to this post by Eric Blake-3
On 10/08/15 05:55, Eric Blake wrote:

> On 08/10/2015 02:18 AM, Juanma wrote:
>
>> Here is another point I find confusing: I thought a "shell builtin" didn't have a separate binary executable file, like 'cd' (which cd => fail),
>
> Actually, POSIX requires that there be a separate 'cd' binary, although
> it does not have to behave the same as the shell builtin.  (About all an
> exec'able cd can do is tell you by exit status whether the builtin cd
> would succeed or fail; or be used for its CDPATH side-effect of printing
> a directory name).
>
> GNU/Linux systems tend to ignore the POSIX requirement of exec'able
> counterparts, although here is how Solaris effectively does it:
>
> $ cat /bin/cd
> #!/bin/sh
> exec $(basename $0) "$@"
> $
>
> and hard-linking that 2-liner to all of the shell builtins where POSIX
> requires to have a non-builtin counterpart.
>
> See also http://austingroupbugs.net/view.php?id=705
>
> It is only the special builtins (such as 'exit') where POSIX does not
> require an exec'able counterpart.

For the record I see this on Fedora 25


$ rpm -q bash
bash-4.3.43-4.fc25.x86_64

$ rpm -ql bash | grep /bin/ | grep -v bash
/usr/bin/alias
/usr/bin/bg
/usr/bin/cd
/usr/bin/command
/usr/bin/fc
/usr/bin/fg
/usr/bin/getopts
/usr/bin/hash
/usr/bin/jobs
/usr/bin/read
/usr/bin/sh
/usr/bin/type
/usr/bin/ulimit
/usr/bin/umask
/usr/bin/unalias
/usr/bin/wait

$ cat /usr/bin/cd
#!/bin/sh
builtin cd "$@"

cheers,
Pádraig

Loading...