Questions about this topic? Sign up to ask in the talk tab.

Difference between revisions of "Unsafe string replacement"

From NetSec
Jump to: navigation, search
 
(40 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 +
{{info|<center>This is a [[programming language]] agnostic [[vulnerability]], effecting any [[application]] which incorrectly uses string replacement to [[sanitize]] data.</center>}}{{prereq|[[programming]]}}
 +
 +
__FORCETOC__
 +
<font size="-2">Special thanks to [[User:hatter|hatter]] for his contributions to this article.</font>
 
=Overview=
 
=Overview=
Unsafe use of string replacement functions to sanitize user input is extremely common. Because string replace ([http://php.net/manual/en/function.str-replace.php str_replace] in PHP) functions only do a single replacement, it is necessary to loop over them until all unsafe characters or strings are removed if you are replacing more than a single character.
+
Unsafe string replacement occurs when a replacement call is used to remove a series of text longer than one character from a string, invoked only once, to sanitize it. Because string replacement ([http://php.net/manual/en/function.str-replace.php str_replace] in [[PHP]], =~ s/// in [[Perl]], etc) functions only do a single replacement, it is necessary to loop over them until all unsafe characters or strings are removed if you are replacing more than a single character. This also applies to replacements powered by regular expressions.
 +
 
 +
=Examples=
  
==Example==
+
==[[PHP]]==
 
A trivial example:
 
A trivial example:
  
Line 19: Line 25:
 
}}
 
}}
  
First an attacker may try a simple [File Inclusion] attack, using '../' to escape. The result:
+
First an attacker may try a simple directory transversal attack, using '../' to escape. The result:
  
 
   Safe filepath is 'etc/passwd'
 
   Safe filepath is 'etc/passwd'
Line 30: Line 36:
  
 
   Safe filepath is '../../../../../../../../../../../etc/passwd'
 
   Safe filepath is '../../../../../../../../../../../etc/passwd'
   [contents of /etc/passwd]
+
   '''[contents of /etc/passwd]'''
  
 
Even if '../' is replaced twice, it can be easily bypassed by using ......///. No matter how many times the replacement is made, the attacker simply needs to nest another layer.
 
Even if '../' is replaced twice, it can be easily bypassed by using ......///. No matter how many times the replacement is made, the attacker simply needs to nest another layer.
Line 41: Line 47:
 
</source>
 
</source>
 
}}
 
}}
 +
  Bypassed by '<<??'
 +
 
{{code
 
{{code
 
|text=
 
|text=
 
<source lang="php">
 
<source lang="php">
str_replace('<script', '', $source);
+
str_replace(array('<script', '<img'), '', $source);
 
</source>
 
</source>
 
}}
 
}}
 +
  Bypassed by '<<imgscript>'
 +
 
{{code
 
{{code
 
|text=
 
|text=
Line 53: Line 63:
 
</source>
 
</source>
 
}}
 
}}
 +
  Bypassed by 'file:/file:///'
  
==Defense==
+
This also affects regular expressions:
 +
{{code|text=<source lang="php">
 +
$string = preg_replace('@[.]{2}\/@','',$string);
 +
</source>}}
  
If one must use str_replace for sensitization (which is strongly advised against), the secure way of doing so would be to loop until no more dangerous strings are found in the source string. The example program implemented in this fashion would look like this:
+
== [[Perl#Regular_Expressions|PCRE]] ==
  
 +
Perl Example:
 +
 +
{{code|text=<source lang="perl">
 +
$string =~ s/[.]{2}\///g;
 +
</source>}}
 +
 +
To test if perl compatible regular expressions are vulnerable to the attack, a small example perl script, ''rexpose.pl'', is attached:
 +
{{code|text=<source lang="perl">
 +
#!/usr/bin/perl
 +
use strict;
 +
use Getopt::Std;
 +
 +
my %opts = ();
 +
getopts('s:p:',\%opts);
 +
 +
my $string  = $opts{s} if ($opts{s}) or usage();
 +
my $pattern;
 +
$pattern = $opts{p} if ($opts{p});
 +
$pattern = '[.]{2}\/' unless (defined $pattern);
 +
$string =~ s/$pattern//g;
 +
print "$string\n";
 +
 +
sub usage
 +
{
 +
    print "rexpose.pl -s [string] [-p pattern]\n";
 +
    print "Regex exposure will display string replacements using s/\$pattern//g.  Pattern defaults to '[.]{2}\\/' to match '../'.\n";
 +
    exit(0);
 +
}
 +
</source>}}
 +
 +
* First, a check is performed to make sure that the string is being stripped at all to being with:
 +
  [user@host ~]$ perl rexpose.pl -s ../../../../../../path/to/file
 +
  path/to/file
 +
* To identify if the replacement is vulnerable, a check using '''....//''' in stead of '''../''' is performed:
 +
  [user@host ~]$ perl rexpose.pl -s ....//....//....//....//....//....//path/to/file
 +
  ../../../../../../path/to/file
 +
 +
== [[Python]] ==
 +
 +
Python also suffers from this issue:
 +
 +
{{code|text=<source lang="python">
 +
>>> string = "....//....//....//"
 +
>>> string.replace("../","")
 +
'../../../'
 +
</source>}}
 +
 +
Example with regular expressions:
 +
 +
{{code|text=<source lang="python">
 +
string = re.sub('[.]{2}\/','',string)
 +
</source>}}
 +
 +
== [[Ruby]] ==
 +
 +
A Ruby example:
 +
 +
{{code|text=<source lang="ruby">
 +
string =~ s/[.]{2}\///g
 +
# OR
 +
# string = string.gsub("../", "")
 +
</source>}}
 +
 +
=Defense=
 +
 +
If one must use str_replace or PCRE for removal of a string (this does not apply to single bytes), it must be done using a loop or called recursively:
 +
 +
==[[PHP]]==
 +
* Loop
 
{{code
 
{{code
 
|text=
 
|text=
 
<source lang="php">
 
<source lang="php">
<?php
+
function safe_str_replace($search, $replace, $subject) {
$filepath = $_GET['file'];
+
        while(strstr($subject, $search) !== FALSE) {
 +
                $subject = str_replace($search, $replace, $subject);
 +
        }
  
$safe_filepath = $filepath;
+
         return $subject;
 
+
while(strstr($safe_filepath, '../') != FALSE) {
+
         $safe_filepath = str_replace('../', '', $safe_filepath);
+
 
}
 
}
 +
</source>
 +
}}
  
echo("Safe filepath is '" . $safe_filepath . "'<br />");
+
* Recursion
include($safe_filepath);
+
{{code
?>
+
|text=
 +
<source lang="php">
 +
function safe_str_replace($search, $replace, $subject) {
 +
        if (strstr($subject, $search) !== FALSE) {
 +
                return safe_str_replace($search,$replace,str_replace($search, $replace, $subject));
 +
        }
 +
        return $subject;
 +
}
 
</source>
 
</source>
 
}}
 
}}
  
However, the use of whitelists of 'positive' regex matching (i.e. does the input match /[a-z]+/) is consider more effective.
+
==[[Perl#Regular_Expressions|PCRE]]==
 +
* When patching ''rexpose.pl'', line 12 contains the following line:
 +
{{code|text=<source lang="perl">
 +
$string =~ s/$pattern//g;
 +
</source>}}
 +
* This can be fixed with a golfed until loop:
 +
{{code|text=<source lang="perl">
 +
$string =~ s/$pattern//g until ($string !~ $pattern);
 +
</source>}}
 +
* This patch was tested:
 +
  [user@host ~]$ perl rexpose.pl -s ....//....//....//....//....//....//path/to/file
 +
  path/to/file
 +
 
 +
==[[Python]]==
 +
 
 +
* Recursion
 +
 
 +
{{code|text=<source lang="python">
 +
def sanitize(str):
 +
  str = str.replace("../", "")
 +
  if "../" in str:
 +
    return sanitize(str)
 +
  return str
 +
</source>}}
 +
 
 +
== [[Ruby]] ==
 +
Ruby sanitizing function:
 +
{{code|text=<source lang="ruby">
 +
def safe_sub(pattern, replacement, string)
 +
  if (string ~ '/#{pattern}/') return safe_sub(pattern, replacement, string.gsub(pattern,replacement))
 +
  return string
 +
end
 +
</source>}}
 +
 
 +
==Whitelisting using PCRE==
 +
It is always best to use whitelisting to sanitize input. This can be done by  using regular expressions to test to be sure that the input contains valid data.  For example, if the developer only wants to refer to a file in the same directory:
 +
{{code|text=<source lang="perl">
 +
my $string = <>;
 +
print $string if ($string ~ /^[\w\d\_\-]+[.][\w\d]{2,4}$/g);
 +
</source>}}
 +
 
 +
 
 +
= Auditing =
 +
{{code|text=<source lang="bash">
 +
find -regextype posix-awk -regex ".*\.e?(rb|py|php)"  -exec grep -HnC2 \\b\\(.*sub\|replace\\)\( '{}' \; &>  string_replacement.txt
 +
</source>}}
 +
 
 +
{{exploitation}}{{programming}}
 +
{{social}}
 +
[[Category:Secure programming]]

Latest revision as of 05:51, 2 December 2012

c3el4.png
This is a programming language agnostic vulnerability, effecting any application which incorrectly uses string replacement to sanitize data.
Unsafe string replacement requires a basic understanding of programming



Special thanks to hatter for his contributions to this article.

Overview

Unsafe string replacement occurs when a replacement call is used to remove a series of text longer than one character from a string, invoked only once, to sanitize it. Because string replacement (str_replace in PHP, =~ s/// in Perl, etc) functions only do a single replacement, it is necessary to loop over them until all unsafe characters or strings are removed if you are replacing more than a single character. This also applies to replacements powered by regular expressions.

Examples

PHP

A trivial example:

 
<?php
$filepath = $_GET['file'];
 
$safe_filepath = str_replace('../', '', $filepath);
 
echo("Safe filepath is '" . $safe_filepath . "'<br />");
include($safe_filepath);
?>
 

First an attacker may try a simple directory transversal attack, using '../' to escape. The result:

 Safe filepath is 'etc/passwd'

No dice, the dangerous string ('../') is dutifully sanitized by str_replace. But, our attacker isn't going to give up yet, now armed with the knowledge that '../' is being filtered out, he may try:

 test.php?file=....//....//....//....//....//....//....//....//....//....//....//....//....//etc/passwd

The result:

 Safe filepath is '../../../../../../../../../../../etc/passwd'
 [contents of /etc/passwd]

Even if '../' is replaced twice, it can be easily bypassed by using ......///. No matter how many times the replacement is made, the attacker simply needs to nest another layer.

Other examples of unsafe uses of string replacement include:

 
str_replace('<?', '', $source);
 Bypassed by '<<??'
 
str_replace(array('<script', '<img'), '', $source);
 
 Bypassed by '<<imgscript>'
 
str_replace('file://', '', $source);
 
 Bypassed by 'file:/file:///'

This also affects regular expressions:

 
$string = preg_replace('@[.]{2}\/@','',$string);
 

PCRE

Perl Example:

 
$string =~ s/[.]{2}\///g;
 

To test if perl compatible regular expressions are vulnerable to the attack, a small example perl script, rexpose.pl, is attached:

 
#!/usr/bin/perl
use strict;
use Getopt::Std;
 
my %opts = ();
getopts('s:p:',\%opts);
 
my $string  = $opts{s} if ($opts{s}) or usage();
my $pattern;
$pattern = $opts{p} if ($opts{p});
$pattern = '[.]{2}\/' unless (defined $pattern);
$string =~ s/$pattern//g;
print "$string\n";
 
sub usage
{
    print "rexpose.pl -s [string] [-p pattern]\n";
    print "Regex exposure will display string replacements using s/\$pattern//g.  Pattern defaults to '[.]{2}\\/' to match '../'.\n";
    exit(0);
}
 
  • First, a check is performed to make sure that the string is being stripped at all to being with:
 [user@host ~]$ perl rexpose.pl -s ../../../../../../path/to/file
 path/to/file
  • To identify if the replacement is vulnerable, a check using ....// in stead of ../ is performed:
 [user@host ~]$ perl rexpose.pl -s ....//....//....//....//....//....//path/to/file
 ../../../../../../path/to/file

Python

Python also suffers from this issue:

 
>>> string = "....//....//....//"
>>> string.replace("../","")
'../../../'
 

Example with regular expressions:

 
string = re.sub('[.]{2}\/','',string)
 

Ruby

A Ruby example:

 
string =~ s/[.]{2}\///g
# OR 
# string = string.gsub("../", "")
 

Defense

If one must use str_replace or PCRE for removal of a string (this does not apply to single bytes), it must be done using a loop or called recursively:

PHP

  • Loop
 
function safe_str_replace($search, $replace, $subject) {
        while(strstr($subject, $search) !== FALSE) {
                $subject = str_replace($search, $replace, $subject);
        }
 
        return $subject;
}
 
  • Recursion
 
function safe_str_replace($search, $replace, $subject) {
        if (strstr($subject, $search) !== FALSE) {
                return safe_str_replace($search,$replace,str_replace($search, $replace, $subject));
        }
        return $subject;
}
 

PCRE

  • When patching rexpose.pl, line 12 contains the following line:
 
$string =~ s/$pattern//g;
 
  • This can be fixed with a golfed until loop:
 
$string =~ s/$pattern//g until ($string !~ $pattern);
 
  • This patch was tested:
 [user@host ~]$ perl rexpose.pl -s ....//....//....//....//....//....//path/to/file
 path/to/file

Python

  • Recursion
 
def sanitize(str):
  str = str.replace("../", "")
  if "../" in str:
    return sanitize(str)
  return str
 

Ruby

Ruby sanitizing function:

 
def safe_sub(pattern, replacement, string)
  if (string ~ '/#{pattern}/') return safe_sub(pattern, replacement, string.gsub(pattern,replacement))
  return string
end
 

Whitelisting using PCRE

It is always best to use whitelisting to sanitize input. This can be done by using regular expressions to test to be sure that the input contains valid data. For example, if the developer only wants to refer to a file in the same directory:

 
my $string = <>;
print $string if ($string ~ /^[\w\d\_\-]+[.][\w\d]{2,4}$/g);
 


Auditing

 
find -regextype posix-awk -regex ".*\.e?(rb|py|php)"  -exec grep -HnC2 \\b\\(.*sub\|replace\\)\( '{}' \; &>  string_replacement.txt
 
Unsafe string replacement is part of a series on exploitation.
<center>
Unsafe string replacement is part of a series on programming.
<center>
</center>