[here string] uncompatible change on here string function

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[here string] uncompatible change on here string function

尹剑虹
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib  -D_GNU_SOURCE -DRECYCLES_PIDS -DDEFAULT_PATH_VALUE='/usr/local/bin:/usr/bin'  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -Wno-parentheses -Wno-format-security
uname output: Linux dhcp-12-238.nay.redhat.com 4.13.9-200.fc26.x86_64 #1 SMP Mon Oct 23 13:52:45 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu

Bash Version: 4.4
Patch Level: 12
Release Status: release

Description:
        The command line to the right of  "<<<"  is no longer cut by IFS

        [yjh@bash4.4 ~]$ var=$(echo -e "foo\t\tbar")
        [yjh@bash4.4 ~]$ read a <<<$var; echo "$a"
        foo bar

        [yjh@bash4.2 ~]$ var=$(echo -e "foo\t\tbar")
        [yjh@bash4.2 ~]$ read a <<<$str; echo "$a"
        foo bar


Repeat-By:
        see section "Description"
Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Greg Wooledge
On Wed, Nov 22, 2017 at 11:22:47AM +0800, 尹剑虹 wrote:
> The command line to the right of  "<<<"  is no longer cut by IFS
>
> [yjh@bash4.4 ~]$ var=$(echo -e "foo\t\tbar")
> [yjh@bash4.4 ~]$ read a <<<$var; echo "$a"
> foo bar
>
> [yjh@bash4.2 ~]$ var=$(echo -e "foo\t\tbar")
> [yjh@bash4.2 ~]$ read a <<<$str; echo "$a"
> foo bar

Quoting failure.

wooledg:~$ bash-4.2
wooledg:~$ var=$'foo\t\tbar'
wooledg:~$ cat <<< "$var"
foo bar
wooledg:~$ read -r a <<< "$var"; declare -p a
declare -- a="foo bar"

When in doubt, always double-quote your variable expansions.  (Except
on the right-hand side of =~ inside a [[ command.)

<http://mywiki.wooledge.org/Quotes> goes into more detail.

Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Peter & Kelly Passchier
On 11/22/2017 08:04 PM, Greg Wooledge wrote:
> wooledg:~$ bash-4.2
> wooledg:~$ var=$'foo\t\tbar'
> wooledg:~$ cat <<< "$var"
> foo bar

I think you are missing the point. He is claiming/reporting:
$ bash-4.4-12
$ var=$'foo\t\tbar'
$ cat <<<$var
foo bar

# Without the quotes, the tabs still get expanded, that should not
# happen, they should collapse into 1 space, this is a bug if true.

Peter

Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Greg Wooledge
On Wed, Nov 22, 2017 at 09:00:09PM +0700, Peter & Kelly Passchier wrote:
> I think you are missing the point. He is claiming/reporting:

Why write code that relies on the idiosyncratic behavior of a shell
when it is given incorrect code?  Whether bash handles incorrect code
in one way or another seems pretty irrelevant, when you should fix the
broken script.

So, an older version of bash used to slap you down hard when you failed
to quote.  And a newer version doesn't; it simply acts as though you
had given the quotes, which allows sloppy code to continue to be written.

One might argue that the newer versions should continue to slap down
the broken code, so that programmers will stop doing it.  But that
hardly seems to qualify as a bug in bash.  Lots of programs swing back
and forth between strictness and lenience.  A counter-argument can be
made that permitting the sloppy code to "work" upholds the maxim of
"be liberal in what you accept".

Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

DJ Mills
In reply to this post by Peter & Kelly Passchier
On Wed, Nov 22, 2017 at 9:00 AM, Peter & Kelly Passchier <
[hidden email]> wrote:

> On 11/22/2017 08:04 PM, Greg Wooledge wrote:
>
>> wooledg:~$ bash-4.2
>> wooledg:~$ var=$'foo\t\tbar'
>> wooledg:~$ cat <<< "$var"
>> foo             bar
>>
>
> I think you are missing the point. He is claiming/reporting:
> $ bash-4.4-12
> $ var=$'foo\t\tbar'
> $ cat <<<$var
> foo             bar
>
> # Without the quotes, the tabs still get expanded, that should not
> # happen, they should collapse into 1 space, this is a bug if true.
>
> Peter
>

The bash 4.4 man page states:

 The word undergoes brace  expansion,  tilde  expansion,  parameter  and
       variable  expansion,  command  substitution,  arithmetic expansion,
and
       quote removal.  Pathname expansion and  word  splitting  are  not
per‐
       formed.   The  result  is  supplied  as a single string, with a
newline
       appended, to the command on its standard input (or file descriptor n
if
       n is specified).

It would seem that this is an intentional change
Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Chet Ramey
On 11/22/17 9:19 AM, DJ Mills wrote:

> On Wed, Nov 22, 2017 at 9:00 AM, Peter & Kelly Passchier <
> [hidden email]> wrote:
>
>> On 11/22/2017 08:04 PM, Greg Wooledge wrote:
>>
>>> wooledg:~$ bash-4.2
>>> wooledg:~$ var=$'foo\t\tbar'
>>> wooledg:~$ cat <<< "$var"
>>> foo             bar
>>>
>>
>> I think you are missing the point. He is claiming/reporting:
>> $ bash-4.4-12
>> $ var=$'foo\t\tbar'
>> $ cat <<<$var
>> foo             bar
>>
>> # Without the quotes, the tabs still get expanded, that should not
>> # happen, they should collapse into 1 space, this is a bug if true.
>>
>> Peter
>>
>
> The bash 4.4 man page states:
>
>  The word undergoes brace  expansion,  tilde  expansion,  parameter  and
>        variable  expansion,  command  substitution,  arithmetic expansion,
> and
>        quote removal.  Pathname expansion and  word  splitting  are  not
> per‐
>        formed.   The  result  is  supplied  as a single string, with a
> newline
>        appended, to the command on its standard input (or file descriptor n
> if
>        n is specified).
>
> It would seem that this is an intentional change

The bash-4.2 man page defers the description of here strings to the
description of here documents. The lines in a here document do not undergo
word splitting.  It was a bug in bash-4.2 that the WORD in a here string
was split.

This finally got fixed in bash-4.4, as described by this CHANGES entry:

z.  Bash no longer splits the expansion of here-strings, as the documentation
    has always said.

I updated the documentation very soon after bash-4.2 was released, back in
2011, and that updated documentation was part of bash-4.3. The bug itself
got fixed in September, 2015 as the result of

http://lists.gnu.org/archive/html/bug-bash/2015-09/msg00022.html

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/

Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Peter & Kelly Passchier
On 11/23/2017 02:23 AM, Chet Ramey wrote:
> The bash-4.2 man page defers the description of here strings to the
> description of here documents. The lines in a here document do not undergo
> word splitting.  It was a bug in bash-4.2 that the WORD in a here string
> was split.
>
> This finally got fixed in bash-4.4, as described by this CHANGES entry:
>
> z.  Bash no longer splits the expansion of here-strings, as the documentation
>      has always said.

I would think it is useful (and according to how things work in general)
to have a different behavior for <<<"$a" and <<<$a

If one wants to have all line-breaks, spaces and tabs preserved, use
<<<"$a" otherwise use <<<$a

Peter

Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Chet Ramey
On 11/22/17 8:17 PM, PePa wrote:

> On 11/23/2017 02:23 AM, Chet Ramey wrote:
>> The bash-4.2 man page defers the description of here strings to the
>> description of here documents. The lines in a here document do not undergo
>> word splitting.  It was a bug in bash-4.2 that the WORD in a here string
>> was split.
>>
>> This finally got fixed in bash-4.4, as described by this CHANGES entry:
>>
>> z.  Bash no longer splits the expansion of here-strings, as the
>> documentation
>>      has always said.
>
> I would think it is useful (and according to how things work in general) to
> have a different behavior for <<<"$a" and <<<$a
>
> If one wants to have all line-breaks, spaces and tabs preserved, use
> <<<"$a" otherwise use <<<$a

Here strings are, as the documentation says, a variant of here documents.
There should be no behavioral difference between

v=$'abc\tdef'

cat <<EOF
$v
EOF

and

cat <<<$v

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/

Reply | Threaded
Open this post in threaded view
|

Re: Re: [here string] uncompatible change on here string function

Greg Wooledge
In reply to this post by Greg Wooledge
On Fri, Nov 24, 2017 at 11:22:56AM +0800, 尹剑虹 wrote:
> It's not the point. the problem is bash-4.4 auto add quote around  $var or $(cmd)  after '<<<'
>
> On bash-4.2 and before  following command always works fine
>   read ignore fstype <<<$(df --output=fstype $mountpoint)
>
> But after Update bash-4.4, our customer's script get fail, do you understand what I mean?

The script is broken.  It happened to work in bash 4.2 because of a bug
in bash 4.2 that cancelled out the bug in the script.

wooledg:~$ df --output=fstype /
Type
ext4

Here is how the script should be written:

{ read; read -r fstype; } < <(df --output=fstype "$mountpoint")

And testing it:

wooledg:~$ mountpoint=/
wooledg:~$ { read; read -r fstype; } < <(df --output=fstype "$mountpoint")
wooledg:~$ declare -p fstype
declare -- fstype="ext4"

Reply | Threaded
Open this post in threaded view
|

Re: [here string] uncompatible change on here string function

Vladimir Marek
> > It's not the point. the problem is bash-4.4 auto add quote around  $var or $(cmd)  after '<<<'
> >
> > On bash-4.2 and before  following command always works fine
> >   read ignore fstype <<<$(df --output=fstype $mountpoint)
> >
> > But after Update bash-4.4, our customer's script get fail, do you understand what I mean?
>
> The script is broken.  It happened to work in bash 4.2 because of a bug
> in bash 4.2 that cancelled out the bug in the script.
>
> wooledg:~$ df --output=fstype /
> Type
> ext4
>
> Here is how the script should be written:
>
> { read; read -r fstype; } < <(df --output=fstype "$mountpoint")
>
> And testing it:
>
> wooledg:~$ mountpoint=/
> wooledg:~$ { read; read -r fstype; } < <(df --output=fstype "$mountpoint")
> wooledg:~$ declare -p fstype
> declare -- fstype="ext4"

Or even
fstype=$( stat --file-system --print "%T\n" "$mountpoint" )

--
        Vlad

Reply | Threaded
Open this post in threaded view
|

回复:[here string] uncompatible change on here string function

尹剑虹
Thanks all.

Now I got it:
The point is the action on bash-4.2 and before deosn't match the action on ksh and zsh.

Jianhong

发自网易邮箱手机版


在2017年11月27日 23:42,Vladimir Marek 写道:

> > It's not the point. the problem is bash-4.4 auto add quote around  $var or $(cmd)  after '<<<'
> >
> > On bash-4.2 and before  following command always works fine
> >   read ignore fstype <<<$(df --output=fstype $mountpoint)
> >
> > But after Update bash-4.4, our customer's script get fail, do you understand what I mean?
>
> The script is broken.  It happened to work in bash 4.2 because of a bug
> in bash 4.2 that cancelled out the bug in the script.
>
> wooledg:~$ df --output=fstype /
> Type
> ext4
>
> Here is how the script should be written:
>
> { read; read -r fstype; } < <(df --output=fstype "$mountpoint")
>
> And testing it:
>
> wooledg:~$ mountpoint=/
> wooledg:~$ { read; read -r fstype; } < <(df --output=fstype "$mountpoint")
> wooledg:~$ declare -p fstype
> declare -- fstype="ext4"

Or even
fstype=$( stat --file-system --print "%T\n" "$mountpoint" )

--
     Vlad