bash errexit shell option does not work in some cases.

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

bash errexit shell option does not work in some cases.

Hyunho Cho

GNU bash, version 5.0.16(1)-release (x86_64-pc-linux-gnu)
 Operating System: Ubuntu 20.04 LTS
 Kernel: Linux 5.4.0-33-generic
 Architecture: x86-64


bash errexit shell option does not work in some cases.
after execution a command, exitcode is set to "1" but shell script does not exited.


#################   WORK CASE   #######################

#!/bin/bash -e                   # errexit option

cat < not_exist.txt             # not exist file

echo end ....

######  result  ######

$ ./test.sh                          # work as expected
./test.sh: line 3: not_exist.txt: No such file or directory


###################   NOT WORK  CASES  #######################

#!/bin/bash -e

{ cat ;} < not_exist.txt  

echo "exitcode : $?"
echo end ....

######  result  ######

$ ./test.sh        
exitcode : 1           # exitcode is set to "1"
./test.sh: line 3: not_exist.txt: No such file or directory
end ....                  # but test.sh does not exited

--------------------------------------------------------------------------------------

#!/bin/bash -e          

while read -r line; do
    echo "$line"
done < not_exist.txt  

echo "exitcode : $?"
echo end.....

######  result  ######

$ ./test.sh
./test.sh: line 5: not_exist.txt: No such file or directory
exitcode : 1        # exitcode is set to "1"
end.....                # but does not exited
Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Eli Schwartz
On 5/31/20 10:32 PM, Hyunho Cho wrote:
>
> GNU bash, version 5.0.16(1)-release (x86_64-pc-linux-gnu)
>  Operating System: Ubuntu 20.04 LTS
>  Kernel: Linux 5.4.0-33-generic
>  Architecture: x86-64
>
>
> bash errexit shell option does not work in some cases.
> after execution a command, exitcode is set to "1" but shell script does not exited.
You're merely the latest person to discover that this applies to you:
http://mywiki.wooledge.org/BashFAQ/105

tl;dr
Don't use errexit, it does not mean what you think it does.

--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Robert Elz
    Date:        Sun, 31 May 2020 22:46:48 -0400
    From:        Eli Schwartz <[hidden email]>
    Message-ID:  <[hidden email]>

While I generally agree with ...

  | Don't use errexit

but not really with:

  | it does not mean what you think it does.

(it does mean what I think it means ...)

in this case:

  | You're merely the latest person to discover that this applies to you:
  | http://mywiki.wooledge.org/BashFAQ/105

doesn't really apply - the cases reported are simply bugs in bash,
a shell implementing -e correctly should exit as it was expected to.

This all results from earlier posix specs where -e and simple commands
were linked - the wording has been improved (still short of perfect)
and simple commands are no longer relevant (other than that exit status
generally originates with the result from a simple command).

I would note that the "echo end..." after the earlier echo that prints
the exit status is poinltless - the echo printing the exit status is all
that is needed, that one exits with status 0, so the following command is
obviously going to run - if the shell were working properly, the first
echo (and anything following) would never be encountered.

All that said, back to the original advice, unless you really know what
you're doing, and really understand the examples in Greg's FAQ page,
then you almost certainly shouldn't be using -e.

kre


Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Eli Schwartz
On 6/1/20 6:28 AM, Robert Elz wrote:

>     Date:        Sun, 31 May 2020 22:46:48 -0400
>     From:        Eli Schwartz <[hidden email]>
>     Message-ID:  <[hidden email]>
>
> While I generally agree with ...
>
>   | Don't use errexit
>
> but not really with:
>
>   | it does not mean what you think it does.
>
> (it does mean what I think it means ...)
(I think shell developers are exempt from this generalization. :p)

> in this case:
>
>   | You're merely the latest person to discover that this applies to you:
>   | http://mywiki.wooledge.org/BashFAQ/105
>
> doesn't really apply - the cases reported are simply bugs in bash,
> a shell implementing -e correctly should exit as it was expected to.
>
> This all results from earlier posix specs where -e and simple commands
> were linked - the wording has been improved (still short of perfect)
> and simple commands are no longer relevant (other than that exit status
> generally originates with the result from a simple command).
As I am a poor user, I could not figure out whether the grammar allowed
or forbade this this result.

So this is actually that unbelievable of unbelievables, a script which
is broken because of set -e and it's actually because set -e has a bug?

Thanks for clarifying.

--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Robert Elz
    Date:        Mon, 1 Jun 2020 09:16:17 -0400
    From:        Eli Schwartz <[hidden email]>
    Message-ID:  <[hidden email]>


  | I could not figure out whether the grammar allowed
  | or forbade this this result.

It isn't the grammar, that would affect syntax, it is the semantics
that matter here.

What the current posix spec says is ...

-e When this option is on, when any command fails (for any of the reasons
   listed in Section 2.8.1 or by returning an exit status greater than zero),
[it exits, except for when ...]

In the examples in question that failed, there was a redirection failure.
of a compound command (grouping in the first, while in the 2nd).  Redirection
failures are amongst those listed in 2.8.1.   None of the exceptions apply
in the example cases, so the shell should have exited.

  | So this is actually that unbelievable of unbelievables, a script which
  | is broken because of set -e and it's actually because set -e has a bug?

I don't know what the original script that led to the bug report was,
nor if it makes sane use of -e (very few scripts do .. -e is best left
for make to use) but the examples show bash "issues".   bash (in
non-posix mode) has some differences wrt when the shell exits, and
doesn't - whether these ones fall into that category or not is not for
me to say.

kre

ps: I know of one script that uses -e in a way that works, it consists
of "set -e" and then a whole long list of simple commands, one after
another, where if one fails, none of the following ones makes sense.
I considered changing it, but as that would have just meant adding
" || exit 1" at the end of every line .. literally every line .. or
possibly making a huge "&&" and-or list (with only and) out of it all,
it seemed like a case where "-e" was actually the right approach.


Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Chet Ramey
In reply to this post by Hyunho Cho
On 5/31/20 10:32 PM, Hyunho Cho wrote:
>
> GNU bash, version 5.0.16(1)-release (x86_64-pc-linux-gnu)
>  Operating System: Ubuntu 20.04 LTS
>  Kernel: Linux 5.4.0-33-generic
>  Architecture: x86-64
>
>
> bash errexit shell option does not work in some cases.
> after execution a command, exitcode is set to "1" but shell script does not exited.

Thanks for the report. These are bugs; they're artifacts from when the
standard said `set -e' worked only on simple commands. This will be fixed
in the next devel branch push.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://tiswww.cwru.edu/~chet/

Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Greg Wooledge
In reply to this post by Robert Elz
On Mon, Jun 01, 2020 at 05:28:20PM +0700, Robert Elz wrote:
> in this case:
>
>   | You're merely the latest person to discover that this applies to you:
>   | http://mywiki.wooledge.org/BashFAQ/105
>
> doesn't really apply - the cases reported are simply bugs in bash,
> a shell implementing -e correctly should exit as it was expected to.

The specification of set -e is not fixed.  It keeps changing.  Bash
implements a version of set -e that was specified at some point in
the past, which included the phrase "simple command".  So, the rules
that apply to compound commands and the rules that apply to simple
commands are not the same.

A more recent POSIX specification has changed the wording, and therefore
bash's implementation is out of date.  A future version of bash will
conform to the new wording, which means it will (once again) break
backward compatibility with older versions of bash (and also with any
other shells that still implement the older specification).

Simply waving this off as a "bug in bash" is quite unfair.

This is part of the set -e nightmare, which is what we keep trying to
get you (and everyone else) to understand.

Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Chet Ramey
On 6/8/20 7:24 AM, Greg Wooledge wrote:

> On Mon, Jun 01, 2020 at 05:28:20PM +0700, Robert Elz wrote:
>> in this case:
>>
>>   | You're merely the latest person to discover that this applies to you:
>>   | http://mywiki.wooledge.org/BashFAQ/105
>>
>> doesn't really apply - the cases reported are simply bugs in bash,
>> a shell implementing -e correctly should exit as it was expected to.
>
> The specification of set -e is not fixed.  It keeps changing.  Bash
> implements a version of set -e that was specified at some point in
> the past, which included the phrase "simple command".  So, the rules
> that apply to compound commands and the rules that apply to simple
> commands are not the same.

This explanation is out of date. It's true that back as recently as
2008, POSIX specified that `set -e' worked only if a simple command
failed. Bash implemented that spec, which was incompatible with other
(and historical) shell implementations.
>
> A more recent POSIX specification has changed the wording, and therefore
> bash's implementation is out of date.  A future version of bash will
> conform to the new wording, which means it will (once again) break
> backward compatibility with older versions of bash (and also with any
> other shells that still implement the older specification).

We hashed out the problems with `set -e' starting back in 2009, and the
standard changed to match historical behavior. This meant that things like
pipelines, (command) subshells, and so on failing would cause the shell to
exit when errexit was enabled. I changed bash to match it starting with
version 4.0, which was followed by much gnashing of teeth and rending of
garments. I'm sure you recall.

No one is arguing that the standard did not change here. In this case,
there was a defect report, (extensive) discussion, an interpretation, and
the standard finally channged in 2013.

> Simply waving this off as a "bug in bash" is quite unfair.

These two cases are, in fact, bugs in bash. They are remnants from the
period when `set -e' only affected simple commands, true, but they are
bugs now nonetheless.

> This is part of the set -e nightmare, which is what we keep trying to
> get you (and everyone else) to understand.

Most of the issue with `set -e' is folks not understanding when it's
not in effect.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://tiswww.cwru.edu/~chet/

Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Eli Schwartz
On 6/8/20 10:02 AM, Chet Ramey wrote:
> On 6/8/20 7:24 AM, Greg Wooledge wrote:
>> This is part of the set -e nightmare, which is what we keep trying to
>> get you (and everyone else) to understand.
>
> Most of the issue with `set -e' is folks not understanding when it's
> not in effect.

Well, Greg does have a pretty good point (which is a point I for one
prioritize in my decision process for "do I personally recommend the use
of set -e") in that another part of the issue is its doing different
things across point releases of a shell.

And that's not counting the times when the issue is people not
understanding when it *is* in effect.

Or people who know exactly when it is and isn't in effect, and have no
issue, but decline to use it because the precise nature of "when it is
and isn't in effect" decreases the usefulness of it sufficiently to make
them decide they're better off implementing error checking a different
way... and *that* is, I think, what Greg is pointing out.

--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User


signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: bash errexit shell option does not work in some cases.

Chet Ramey
On 6/8/20 10:44 AM, Eli Schwartz wrote:

> Well, Greg does have a pretty good point (which is a point I for one
> prioritize in my decision process for "do I personally recommend the use
> of set -e") in that another part of the issue is its doing different
> things across point releases of a shell.

It last changed in 2009. Except for MacOS X, which is frozen in amber,
people should be using a version with the current semantics by now. If you
don't want things to change across releases, you have choices about which
version to use.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://tiswww.cwru.edu/~chet/


signature.asc (242 bytes) Download Attachment