bash 4.4 null byte warning!!

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

bash 4.4 null byte warning!!

emlyn.jose@wipro.com

Hi,

 

We are trying to use the bash 4.4 downloaded from http://ftp.gnu.org/gnu/bash/ on our RHEL 6.2 machine.

But while using a script that has this command :

FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6 -print0`

It throws a warning as below:

bash: warning: command substitution: ignored null byte in input

 

 

Is there any patch available for this or is there any way to make bash silently drop this warning(which has been the behavior on the lower versions of bash)??

 

We cannot find this new bash version on the Redhat Portal.

 

Regards,

Emlyn Jose

Wipro Technologies

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: bash 4.4 null byte warning!!

Greg Wooledge
On Thu, Apr 06, 2017 at 07:47:35AM +0000, [hidden email] wrote:
> FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6 -print0`
> It throws a warning as below:
>
> bash: warning: command substitution: ignored null byte in input

Your command is broken, and bash is warning you of this.  Why do you want
to silence the warning instead of fixing the command?

If your find returns a single result, you get a filename followed by NUL,
and the NUL is discarded, leaving you a filename inside the command
substitution (and therefore the variable).  You could achieve the EXACT
same result by dropping the -print0.  Then, if you get a single result
from find, you get the filename followed by newline, and the command
substitution discards the trailing newline, leaving you just the filename
in the command substitution and therefore the variable.

On the other hand, if your find command returns MULTIPLE results, then
you have file1\000file2\000 and the command substitution drops the NUL
bytes, leaving you with file1file2 in your variable.  At least if you
dropped the -print0 you would have file1\nfile2 in your variable, which
is still wrong, but not as wrong as file1file2.

> Is there any patch available for this or is there any way to make bash silently drop this warning(which has been the behavior on the lower versions of bash)??

If you REALLY want to continue to run a broken command, you can explicitly
use tr to remove the NULs before the command substitution ever sees
them.

var=$(find ... -print0 | tr -d '\0')

That will prevent the warning and allow you to continue running a broken
script without letting your coworkers catch on to the bugs (or whatever
your issue with the warning is).

If you actually want to FIX the script, the results of find should be
read into an array, since there can be more than one.  Every place you
use your variable currently, you'd have to rewrite that to handle an
array with potentially multiple elements.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: bash 4.4 null byte warning!!

Chet Ramey
In reply to this post by emlyn.jose@wipro.com
On 4/6/17 3:47 AM, [hidden email] wrote:

> Hi,
>
>  
>
> We are trying to use the bash 4.4 downloaded from
> http://ftp.gnu.org/gnu/bash/ on our RHEL 6.2 machine.
>
> But while using a script that has this command :
>
> FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6-print0`
>
> It throws a warning as below:
>
> bash:warning:command substitution:ignored null byte ininput

Yes.  It drops the null bytes because C strings can't handle them.  I
received bug reports asking why bash silently transforms the command
substitution output and added the warning while not changing the behavior.

If you want to suppress the warning, you'll have to edit the source and
rebuild.

This does beg the question of why you're using -print0 without something
on the receiving end to handle the null bytes.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: bash 4.4 null byte warning!!

emlyn.jose@wipro.com
In reply to this post by Greg Wooledge
Hi Greg/Chet Ramey,
 
Thank  you so much for the response! It gave me an idea on what has to be done next.
 
We have been using the same script all these years and had no issues of broken commands when used with the lower versions of bash which is why I have been thinking whether to rewrite the script for the new bash(4.4) or edit the bash source.
 
The reason behind using –print0 is to use the resultant output to find a binary
 
Below are the lines of code followed by the find command:
 
FIND_RPM="$(find $linIdx -type d -name "${2}" -print0 2>/dev/null)"
        [ -n "$FIND_RPM" ] && \
            [ -s "${FIND_RPM}/bin" ] && \
            bdt_msg 4 "[D] Found %s in %s component area." "$2" "$linIdx" && \
               
               
 
>> If you want to suppress the warning, you'll have to edit the source and rebuild.
Do you have a guide on how to do this?
 
Or else I have been planning to add tr to remove the nuls as suggested by Greg.
 
Regards,
Emlyn Jose.
 
---------------------------------------------------------------------------------------------------------
-----Original Message-------------------------------------------------------------------------------
From: Chet Ramey [[hidden email]]
Sent: Saturday, April 8, 2017 7:48 AM
To: Emlyn Jose (GIS); [hidden email]
Cc: [hidden email]
Subject: Re: bash 4.4 null byte warning!!
 
On 4/6/17 3:47 AM, [hidden email] wrote:
> Hi,
>
>
>
> We are trying to use the bash 4.4 downloaded from
> http://ftp.gnu.org/gnu/bash/ on our RHEL 6.2 machine.
>
> But while using a script that has this command :
>
> FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6-print0`
>
> It throws a warning as below:
>
> bash:warning:command substitution:ignored null byte ininput
 
Yes.  It drops the null bytes because C strings can't handle them.  I received bug reports asking why bash silently transforms the command substitution output and added the warning while not changing the behavior.
 
If you want to suppress the warning, you'll have to edit the source and rebuild.
 
This does beg the question of why you're using -print0 without something on the receiving end to handle the null bytes.
 
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/
 
 
--------------------------------------------------------------------------------------------------------------------------------
-----Original Message------------------------------------------------------------------------------------------------------
From: Greg Wooledge [[hidden email]]
Sent: Thursday, April 6, 2017 8:15 PM
To: Emlyn Jose (GIS)
Cc: [hidden email]
Subject: Re: bash 4.4 null byte warning!!
 
On Thu, Apr 06, 2017 at 07:47:35AM +0000, [hidden email] wrote:
> FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6 -print0`
> It throws a warning as below:
>
> bash: warning: command substitution: ignored null byte in input
 
Your command is broken, and bash is warning you of this.  Why do you want to silence the warning instead of fixing the command?
 
If your find returns a single result, you get a filename followed by NUL, and the NUL is discarded, leaving you a filename inside the command substitution (and therefore the variable).  You could achieve the EXACT same result by dropping the -print0.  Then, if you get a single result from find, you get the filename followed by newline, and the command substitution discards the trailing newline, leaving you just the filename in the command substitution and therefore the variable.
 
On the other hand, if your find command returns MULTIPLE results, then you have file1\000file2\000 and the command substitution drops the NUL bytes, leaving you with file1file2 in your variable.  At least if you dropped the -print0 you would have file1\nfile2 in your variable, which is still wrong, but not as wrong as file1file2.
 
> Is there any patch available for this or is there any way to make bash silently drop this warning(which has been the behavior on the lower versions of bash)??
 
If you REALLY want to continue to run a broken command, you can explicitly use tr to remove the NULs before the command substitution ever sees them.
 
var=$(find ... -print0 | tr -d '\0')
 
That will prevent the warning and allow you to continue running a broken script without letting your coworkers catch on to the bugs (or whatever your issue with the warning is).
 
If you actually want to FIX the script, the results of find should be read into an array, since there can be more than one.  Every place you use your variable currently, you'd have to rewrite that to handle an array with potentially multiple elements.
 
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: bash 4.4 null byte warning!!

Reuti

> Am 10.04.2017 um 07:08 schrieb [hidden email]:
>
> Hi Greg/Chet Ramey,
>
> Thank  you so much for the response! It gave me an idea on what has to be done next.
>
> We have been using the same script all these years and had no issues of broken commands when used with the lower versions of bash which is why I have been thinking whether to rewrite the script for the new bash(4.4) or edit the bash source.
>
> The reason behind using –print0 is to use the resultant output to find a binary
>
> Below are the lines of code followed by the find command:
>
> FIND_RPM="$(find $linIdx -type d -name "${2}" -print0 2>/dev/null)"
Why using -print0 here? You can fill the $FIND_RPM variable also without it, the \n is not part of the variable afterwards but you get one long line if more than one entry is found (and this couldn't be used below by appending /bin).

FIND_RPM="$(find $linIdx -type d -name "${2}" -print -quit)"

if you want only the first one found. And why not searching directly for "${2}/bin".

-- Reuti


>         [ -n "$FIND_RPM" ] && \
>             [ -s "${FIND_RPM}/bin" ] && \
>             bdt_msg 4 "[D] Found %s in %s component area." "$2" "$linIdx" && \
>                 …
>                 …
>
> >> If you want to suppress the warning, you'll have to edit the source and rebuild.
> Do you have a guide on how to do this?
>
> Or else I have been planning to add tr to remove the nuls as suggested by Greg.
>
> Regards,
> Emlyn Jose.
>
> ---------------------------------------------------------------------------------------------------------
> -----Original Message-------------------------------------------------------------------------------
> From: Chet Ramey [mailto:[hidden email]]
> Sent: Saturday, April 8, 2017 7:48 AM
> To: Emlyn Jose (GIS); [hidden email]
> Cc: [hidden email]
> Subject: Re: bash 4.4 null byte warning!!
>
> On 4/6/17 3:47 AM, [hidden email] wrote:
> > Hi,
> >
> >
> >
> > We are trying to use the bash 4.4 downloaded from
> > http://ftp.gnu.org/gnu/bash/ on our RHEL 6.2 machine.
> >
> > But while using a script that has this command :
> >
> > FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6-print0`
> >
> > It throws a warning as below:
> >
> > bash:warning:command substitution:ignored null byte ininput
>
> Yes.  It drops the null bytes because C strings can't handle them.  I received bug reports asking why bash silently transforms the command substitution output and added the warning while not changing the behavior.
>
> If you want to suppress the warning, you'll have to edit the source and rebuild.
>
> This does beg the question of why you're using -print0 without something on the receiving end to handle the null bytes.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>                  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/
>
>
> --------------------------------------------------------------------------------------------------------------------------------
> -----Original Message------------------------------------------------------------------------------------------------------
> From: Greg Wooledge [mailto:[hidden email]]
> Sent: Thursday, April 6, 2017 8:15 PM
> To: Emlyn Jose (GIS)
> Cc: [hidden email]
> Subject: Re: bash 4.4 null byte warning!!
>
> On Thu, Apr 06, 2017 at 07:47:35AM +0000, [hidden email] wrote:
> > FIND_RPM=`find /opt/RPM/components -type d -name enum-1.1.6 -print0`
> > It throws a warning as below:
> >
> > bash: warning: command substitution: ignored null byte in input
>
> Your command is broken, and bash is warning you of this.  Why do you want to silence the warning instead of fixing the command?
>
> If your find returns a single result, you get a filename followed by NUL, and the NUL is discarded, leaving you a filename inside the command substitution (and therefore the variable).  You could achieve the EXACT same result by dropping the -print0.  Then, if you get a single result from find, you get the filename followed by newline, and the command substitution discards the trailing newline, leaving you just the filename in the command substitution and therefore the variable.
>
> On the other hand, if your find command returns MULTIPLE results, then you have file1\000file2\000 and the command substitution drops the NUL bytes, leaving you with file1file2 in your variable.  At least if you dropped the -print0 you would have file1\nfile2 in your variable, which is still wrong, but not as wrong as file1file2.
>
> > Is there any patch available for this or is there any way to make bash silently drop this warning(which has been the behavior on the lower versions of bash)??
>
> If you REALLY want to continue to run a broken command, you can explicitly use tr to remove the NULs before the command substitution ever sees them.
>
> var=$(find ... -print0 | tr -d '\0')
>
> That will prevent the warning and allow you to continue running a broken script without letting your coworkers catch on to the bugs (or whatever your issue with the warning is).
>
> If you actually want to FIX the script, the results of find should be read into an array, since there can be more than one.  Every place you use your variable currently, you'd have to rewrite that to handle an array with potentially multiple elements.
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.www.wipro.com


signature.asc (210 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: bash 4.4 null byte warning!!

Greg Wooledge
In reply to this post by emlyn.jose@wipro.com
On Mon, Apr 10, 2017 at 05:08:10AM +0000, [hidden email] wrote:
>  FIND_RPM="$(find $linIdx -type d -name "${2}" -print0 2>/dev/null)"
>         [ -n "$FIND_RPM" ] && \
>             [ -s "${FIND_RPM}/bin" ] && \
>             bdt_msg 4 "[D] Found %s in %s component area." "$2" "$linIdx" && \

"$FIND_RPM/bin" indicates that your code assumes find will only return
a single directory name.  If find returned two things, you would end up
constructing a path like /thing/one/thing/two/bin.

-print0 makes find print a NUL byte after each filename found.  It's used
to safely serialize filenames into an output stream, since filenames may
contain spaces, tabs, newlines, and so on.  In your case, it's not actually
helping you.  If find returns two or more things, your script is already
broken, and the NUL delimiter between filenames won't save you.

The whole time you have been using this script in the past, find has been
adding a NUL byte (because of -print0) and then bash's command substitution
has been silently removing it.  The only thing that changed in 4.4 is now,
when bash removes the NUL, it issues a warning message.  That's all.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: bash 4.4 null byte warning!!

Chet Ramey
In reply to this post by emlyn.jose@wipro.com
On 4/10/17 1:08 AM, [hidden email] wrote:

>>>If you want to suppress the warning, you'll have to edit the source and
> rebuild.
> Do you have a guide on how to do this?

Change the #if 1 on line 5966 of subst.c to #if 0.  Then follow the
directions in INSTALL, which basically consist of the usual
`configure ; make ; make install' idiom common to GNU software.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: bash 4.4 null byte warning!!

emlyn.jose@wipro.com
Hi guys!

My organization is receiving repeated alerts for using my mail id here : https://mail-archive.com/bug-bash@.../msg19561.html

Could you please remove my maid id?

Thank you!

Regards,
Emlyn Jose.

-----Original Message-----
From: Chet Ramey [mailto:[hidden email]]
Sent: Tuesday, April 11, 2017 4:29 AM
To: Emlyn Jose (GIS) <[hidden email]>; Greg Wooledge <[hidden email]>
Cc: [hidden email]; [hidden email]
Subject: Re: bash 4.4 null byte warning!!

** This mail has been sent from an external source **

On 4/10/17 1:08 AM, [hidden email] wrote:

>>>If you want to suppress the warning, you'll have to edit the source
>>>and
> rebuild.
> Do you have a guide on how to do this?

Change the #if 1 on line 5966 of subst.c to #if 0.  Then follow the directions in INSTALL, which basically consist of the usual `configure ; make ; make install' idiom common to GNU software.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    [hidden email]    http://cnswww.cns.cwru.edu/~chet/
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: bash 4.4 null byte warning!!

Pierre Gaston
On Thu, Aug 3, 2017 at 1:56 PM, [hidden email] <[hidden email]>
wrote:

> Hi guys!
>
> My organization is receiving repeated alerts for using my mail id here :
> https://mail-archive.com/bug-bash@.../msg19561.html
>
> Could you please remove my maid id?
>
> Thank you!
>
> Regards,
> Emlyn Jose.
>
>
This is a public mailing list, moreover mirrored on usenet so there are
many, many copies of these emails that are publicly available about about
which the administrators of this list can do little about.

I'm afraid the site you point to is just one of these, it's a third party
site that makes the mailing list available to everybody.
You should try to contact them see: https://www.mail-archive.com/
faq.html#delete
Loading...