Undesigned: Amazon S3 PHP Class

Blog

Amazon S3 PHP Class

Monday October 22, 2007

This class is a standalone Amazon S3 REST implementation for PHP 5.2.x (using CURL), that supports large file uploads and doesn't require PEAR.

26th/Sep/2008: Buy some karma! Be the first person to make a donation.

Checkout latest source:

svn checkout http://amazon-s3-php-class.googlecode.com/svn/trunk/ s3-php

Download source:

Download latest version (0.3.4 - 11th Sep 2008)

Example usage (see example.php for a more detailed example):


$s3 = new S3('accessKey', 'secretKey');
$s3->putBucket('bucket', S3::ACL_PUBLIC_READ);
$s3->putObjectFile('file.doc', 'bucket', 'docs/file.doc', S3::ACL_PUBLIC_READ);
$s3->deleteObject('bucket', 'docs/file.doc');

Writing and reading resources:


# Upload an object from a resource (requires bytesize):
$s3->putObject($s3->inputResource(fopen($file, 'rb'), filesize($file)), $bucketName, $uploadName, S3::ACL_PUBLIC_READ);

# Download an object to a resource:
$s3->getObject($bucketName, $uploadName, fopen($savefile, 'wb'));

Known Issues:

Files larger than 2GB are not supported on 32 bit systems due to PHP's signed integer problem

More Information:

Changelog

0.3.4 (2008-09-11):
Small updates and support for 0 byte data
0.3.3 (2008-07-30):
Fixed minor getBucket() bugs, added delimiter
0.3.2 (2008-07-12):
Fixed ?logging bug and changed putObject() to accept request headers (thanks to Angelo Mandato for providing details)
0.3.1 (2008-06-20):
Added copyObject() and updated README.txt
0.2.9 (2008-06-20):
Added getBucketLocation() and a location parameter to putBucket() for EU buckets
0.2.8 (2008-06-06):
Added sort() for AMZ meta headers (thanks Malone)
0.2.7 (2008-05-30):
Commented out uncommented var_dump() (thanks Chris)
0.2.6 (2008-05-29):
Replaced isset() with array_key_exists() since isset() was ignoring null parameters (thanks Joe)
0.2.5 (2008-05-29):
Added support for ?location and ?torrent parameters (thanks Joe)
0.2.4 (2008-05-27):
Fix for ?prefix bug (thanks Lucas)
0.2.3 (2008-05-04):
Small bug fixes, updated examples and comments, moved source to SVN
0.2.2 (2008-05-02):
Added S3 wrapper example
0.2.1 (2008-04-28):
First 0.2 beta (0.1 development discontinued)

Comments

hi I really like your s3 php class, very clean abstracted code. Do you have any ideas on streaming? instead of put, I saw a couple different sites, with examples, like, this one, http://www.missiondata.com/blog/linux/49/s3-streaming-with-php/

Please let me know if you enhance your class to do this

-- Jimmy Ho on 30 November 2007

How would one go about uploading a folder? Apart from putObjectFile into a folder subdirectory? Is there a way to take a whole folder, its files, and subfolders at once?

-- Scott on 26 January 2008

Just downloaded and looked at your source, and I like it… The example.php works perfectly. Thank you very much!

-- puzz on 5 February 2008

Hey! Awesome script, very useful. I miss a couple of things though:

A listBuckets function, to check which buckets exists.

A checkObject function that checks if an object with the (file)name exists, without having to get a whole object.

Maybe these things can be done allready, but I can’t see it. Thanks!

-- Hans on 7 February 2008

thanks for the class, been trying to find one that didn’t use pear. New function suggestion would be a getBuckets() function to get a listing of all the buckets you have created.

-- Todd on 15 February 2008

@Jimmy: Streaming like you mentioned probably is possible, will have a look at it sometime.

@Hans: There’s the listBuckets() function you wanted. I’m not sure if a checkObject() could be implemented.

@Scott: You’d need to iterate through the directory uploading each file.

@Todd: Yeah, the PEAR requirement is what made me write this class. listBuckets() there for you too!

Hope you all find it useful. If you use it for something cool, drop me a line and let me know!

-- Don on 18 February 2008

Good stuff, thanks Don!

-- Hans on 19 February 2008

RE: checkObject()
there is a function in the api called HEAD that will fetch the metadata of a file. (Using Rest API > Operations on objects > HEAD object) That may give you what you need as it will give a file not found error if the name doesn’t exist.

And Don, thanks for the update

-- Todd on 22 February 2008

Some kind of streaming large files would be a great addition to this script. I seem to get time outs when trying files even as ‘small’ as 13-15mb. Perhaps that’s more of a local (hosting) issue than scripting?

-- Hans on 25 February 2008

@Hans: Are you getting timeouts on getObject() or putObjectFile()? The putObjectFile() method doesn’t place the entire file into memory when uploading, but getObject() on the other hand does. I haven’t had any problems uploading files as big as 150MB and it usually transfers at about 40kb/s.

-- Don on 25 February 2008

I’m testing this on my localhost and it’s doing fine except for the the fact that I can’t get it to upload a 250k image. I can upload an image that is around 60k and smaller but I am not sure why I cannot upload this particular file. Do you have any ideas or suggestions?

-- Rich on 28 February 2008

tis is nice, is there a way to create folders?

-- Fuerchti on 28 February 2008

@Don: The timeouts occur with putObjectFile(). Have you tried the script on a shared hosting space? It might be something there..

Is there someway I could debug this?

Thanks for your support!

-- Hans on 29 February 2008

When I try to upload large files it will just stop after a couple minutes. It gives no error.

-- Nic on 29 February 2008

Thanks for this, haven’t used to the class yet but intend to. Just noticed a spelling error with method getObjecInfo – shouldn’t it be getObjectInfo?

-- Adam on 10 March 2008

@Adam: Thanks mate!

@Nic, @Hans, @Rich: Not sure why you’re experiencing timeouts, might require a setopt for CURLOPT_TIMEOUT or something… I’ll look into it sometime.

-- Don on 10 March 2008

Example doesn’t work for me. Error as follows:

sweeney@desktop% ./example.php

Parse error: parse error, expecting `’(’‘ in /Users/sweeney/s3/s3-php5-curl/example.php on line 39
sweeney@desktop% php -v
PHP 5.2.5 (cli) (built: Mar 11 2008 12:32:56)
Copyright © 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright © 1998-2007 Zend Technologies
sweeney@desktop%

Environment is MacOS 10.4; hopefully this is a simple problem. Example.php is as delivered, but with real key values.

-- Tony Sweeney on 11 March 2008

@Tony: Works fine for me. Perhaps you modified the file without noticing?

-- Don on 11 March 2008

@Don: problem was pilot error — had stale PHP version laying around somewhere.

Tony.

-- Tony Sweeney on 11 March 2008

don, great script! everything is working quite nicely. quick question — how do you get a private object back from S3 from the command line?

-- Michael on 13 March 2008

@Michael: Use getObject()

-- Don on 13 March 2008

Great work… lean and mean. I echo what everyone else said about the PEAR bloat.

But… you didn’t mention that it needed Simple XML. Worse, you used an @ in the call: @simplexml_load_string($data\[1\]) : $data\[1\];

(Backslashes are because your web site want to make superscripts from the array indexes.)

Result was absolute silence.

Ouch!

But, it’s such a great piece of work that I forgive you.

—Marc

-- Marc Rochkind on 14 March 2008

I Studied your script it was nice.
I have one question for you there a way to create folders inside bucket.
I’ll be thankful if you reply me.

-- Bivek on 25 March 2008

nice work. Good clean code and easy to use.

-- flungabunga on 26 March 2008

Can any one tell me why use cURL instead of PEAR HTTP ?

-- Akshar on 31 March 2008

@Bivek: Not sure – and you’re not the first person to ask. Haven’t tried this yet but it’s probably possible.

@Ashkar: The PEAR requirement everyone else is talking about is for the HMAC-SHA1 key generation. CURL simply provides a nice HTTP client interface.

-- Don on 31 March 2008

Thanks for the class, works great. Does the putObjectFile place files in folders under bucket? Only seems to work putting files in the main bucket.

Thanks!

-- Del on 5 April 2008

Need help while creating a bucket. No matter what bucketName I give it sends same error Unable to create bucket (it may already exist and/or be owned by someone else)<br><br> Does anyone have a solution? Thanks in advance

-- Sachin on 7 April 2008

Thanks for writing this! Question – if a bucket has more than 1000 objects in it, how do you list them all?

-- fletch on 11 April 2008

This doesn’t work on 32 bit systems for files larger than ~2.1 GB. There are two main reasons: 1) php is_file function fails on files over this size 2) php md5_file function fails on files over this size

-- ChrisN on 13 April 2008

When I’m trying to upload a image with the example.php, then I can’t VIEW it in Firefox (Firefox will download the file), but I can view it in Safari and Opera. What’s wrong?

-- Anon on 13 April 2008

Well done and thanks a lot. Will have to backport it to PHP 5.1.x as not all servers in our cluster are on 5.2 yet.. but definitely a perfect class – and all that without PEAR. :-)

-- Rob Kaper on 18 April 2008

Just wondering if anyone else is experiencing that the API requests take upwards of a minute to complete?

The slowness comes from this line in S3Object::getResponse:
$data = explode(”\r\n\r\n”, curl_exec($curlReq), 2);

I have PHP 5.2.5, and curl.

Perhaps it is AWS’s fault? Any other ideas?

-- Jonathan Lipps on 24 April 2008

I love your S3 package, but upon using it, I noticed that it didn’t print error messages, didn’t handle truncation (over 1000 keys in a bucket). I’ve modified it to do so, and if you’re interested, I’d like to send it to you so others can benefit from this. Can you email me how I can do this? Thanks.

-- Corey on 26 April 2008

@Sachin, @Jonathan, @Corey, @fletch: Try the 0.2 beta :)

@Rob: Might even work on 5.1 I don’t think there is any reason it wouldn’t.

-- Don on 29 April 2008

GREAT WORK!!!

Thanks from Spain.

The best and simplest class for S3

-- EduT on 2 May 2008

Great work! I use this script in some of my personal projects and it works great.

One of my projects required the use of PHP4 without any PEAR packages and figuring out how to create HMAC signatures was difficult. In case anyone else needs to do this, I documented my results here:

http://blog.raamdev.com/2008/05/03/amazon-s3-hmac-signatures-without-pear-or-php5/

-- Raam Dev on 4 May 2008

Does svn co require user/password?

-- Jason on 5 May 2008

I am using latest beta release 0.2.3 I am getting following error :

==
Warning: S3::putBucket(Rajantestbucket): [SignatureDoesNotMatch] The request sig
nature we calculated does not match the signature you provided. Check your key a
nd signing method. in C:\xampp\htdocs\S3.php on line 188
=====
I have checked both Accesskey and Secretkey, they seems to be ok.

Do you know where I am going wrong ?

——-

<?php

require_once(‘s3.php’);

$bucketName = “Rajantestbucket”;
$s3 = new S3(“ABCDEFGHI”, “XYHDSDSJHD’KDSLKDSL”);
if( $s3->putBucket( $bucketName, S3::ACL_PUBLIC_READ) ) echo “Bucket : “ . $bucketName . “Created”;

echo “S3::listBucket(): “.print_r($s3->listBuckets(),1). “\n”;
// $s3->putObjectFile(‘file.doc’, ‘bucket’, ‘docs/file.doc’, S3::ACL_PUBLIC_READ);
// $s3->deleteObject(‘bucket’, ‘docs/file.doc’);

-- Rajan on 7 May 2008

@Rajan,

From the AWS documentation:

Bucket names can only contain lowercase letters, numbers, periods (.), underscores (_), and dashes (-).

-- Don on 7 May 2008

Which is the best method for uploading a file from a form?

Put an object from a resource? from a file? from a string?

Thanks!

The file is uploaded first to my server using multipart form data.

-- Edu on 9 May 2008

Great class. Love the putObjectFile method. I used to create an object, check for type and upload the object…this method does it all. Very well written. Thank you!!

-- Ren on 13 May 2008

Hi,

you wrote:

@Scott: You’d need to iterate through the directory uploading each file.

can you explain this. i’m sorry, but i don’t understand how to make this iteration – in what place.

-- Dawid Gatti on 24 May 2008

OK i figure it out. if you are wondering how to create a directory in a bucket this is how you do it.

$s3->putObjectFile($uploadFile, $bucketName, ‘io/’.$uploadFile, S3::ACL_PUBLIC_READ)

the “io” is the new directory.

-- Dawid Gatti on 24 May 2008

Hello,

I’ve got your script working, and I like it a lot. Thanks!

But whenever I try to specify a $prefix to getBucket(), I get an error saying “[SignatureDoesNotMatch] The request signature we calculated does not match the signature you provided. Check your key and signing method.”

I tried looking through your code, but I can’t find what might cause this. Do you know why this might occur?

-- lucas on 27 May 2008

@Lucas: Try the latest source

-- Don on 27 May 2008

Don, i have the same problem whit the latest code.

the funny thing is that i can upload jpg, wav, zip, pdf, txt, cab

but i can’t upload mov, mp3, iso

of course i didn’t test all the formats but you can see that something is wrong. the size doesn’t mater. i can upload a bigger jpg file and have no problem, but uploading a smaller mp3, mov etc generate this problem about signature.

any idea?

-- Dawid Gatti on 28 May 2008

around line 773, I change a little for getAccessControl to work properly. if (isset($this->parameters[‘acl’]) || $this->parameters[‘acl’]==null) { $this->resource .= $query; }

-- joe on 28 May 2008

my last post has some bug too. To make the object work, some changes are still needed to get the correct resource value for the lasted version. I change to the following and it seems working:
$appendArray = array(‘acl’,‘location’,‘torrent’); foreach($appendArray as $append) { if(isset($this->parameters[$append])) { $this->resource .= “?$append”; } }

-- joe on 28 May 2008

in the setAccessControlPolicy(),

the new S3Request has a bug missing $uri;
$rest = new S3Request(‘PUT’, $bucket, $uri);

also, it seems I have to set acl value as 1 to make it work and attached to resource instead of null.
$rest->setParameter(‘acl’, 1);

-- joe on 29 May 2008

@Joe: Thanks for the heads up, didn’t notice that isset() would ignore null array values there…

-- Don on 29 May 2008

I still have the same problem whit 2.6

string(11) “test.foo.pl” string(15) “/klawaitura.mov” string(27) “/test.foo.pl/klawaitura.mov” string(51) “https://test.foo.pl.s3.amazonaws.com/klawaitura.mov”
Warning: S3::putObject(): [SignatureDoesNotMatch] The request signature we calculated does not match the signature you provided. Check your key and signing method. in /home/users/dawidgatti2/public_html/s3/S3up.php on line 312

-- Dawid Gatti on 30 May 2008

WOW!, this is EXACTLY what i’ve been looking for! I just uploaded a 14mb file at 120kb/s (which is my max), and it only took me about 30 seconds to look at the example to find out how, thanks!

-- Nick on 30 May 2008

Implemented in less that an hour…awesome work. Quick question though: when I call $s3->putObjectFile(), why does it print out the filename stuff afterwards? EG:

string(12) “bucket” string(25) “/folder1/folder2/file.flv” string(38) “/bucket/folder1/folder2/file.flv” string(62) “https://bucket.s3.amazonaws.com/folder1/folder2/file.flv”

With bucket and folder being the actual buckets and folders? Is there a print function happening somewhere upon successful upload? Thanks!

-- Chris on 30 May 2008

@Chris: There’s a var_dump in there for debugging, just comment it out.

-- Don on 30 May 2008

Hello

I am trying to delete a bucket which is listed as

b96a4960ce964987b1a381bb35ca33-Test

but went I use the $s3->deleteBucket command all I get is access-denied or can not find bucket. What am I doing wrong

If I create a buket like rob123 that delete fine it just can not find out how to delete these ones. Please help

-- Rob on 3 June 2008

@Rob: “Bucket names can only contain lowercase letters, numbers, periods (.), underscores (_), and dashes (-).”

-- Don on 4 June 2008

Hi, I must thank you for this because Amazon’s documentation is useless, I was up and running with this in no time. You have been a huge help!

I’m having a problem though which I was wondering if you had experienced, the script hangs when uploading very small objects, such as a 1×1 spacer gif, or a browser-dependent stylesheet with just a line or two. The file does appear in the bucket, but then it dies.

I added a CURL timeout which seemed sensible anyway but to no avail.

-- SJ on 4 June 2008

Sorry Don

That is not the case. As this is the program that created them was Jungle Disk. Therefore remember that I use the list option to find the, out.

Therefore they are already created I wish to delete them.

Rob

-- Rob on 5 June 2008

I haven’t changed any code and i’m getting this error whenever i navigate to the s3.php page

Parse error: syntax error, unexpected T_CONST, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or ‘}’ in /homepages/31/d204952132/htdocs/iarp/scripts/s3/S3.php on line 37

Please add me on MSN iarp@cogeco.ca if you’ve anyidea what’s wrong.

-- Ian on 6 June 2008

Great class, it’s really saved me a lot of time.

But I was getting [SignatureDoesNotMatch] errors when I sent certain metadata. The docs say you have to “Sort the collection of headers lexicographically by header name” before signing them. So I added “sort($amz);” before line 791 of S3.php, and that fixed the problem.

-- malone on 6 June 2008

@Ian: This class requires PHP 5 and you are using PHP 4.

@Malone: Thanks for the heads up.

-- Don on 6 June 2008

Wow, quick update!

I’m also having problems uploading empty (zero byte) files. I’m creating a backup system and it’s important that those files are stored, even though they have no content.

I’ve managed to upload empty files using another tool, so it seems to be possible. But when I try to do it with your class I get [NotImplemented]. Also, if i try to upload an empty string it fails silently without an error.

-- malone on 6 June 2008

@Malone: I’m not sure about that… Could have something to do with CURL… Guess one
option would be to insert some sort of single byte character into it?

-- Don on 6 June 2008

Works beautifully, many thanks :)

How hard would it be to add in a parameter to putBucket to specify the location of the bucket? Right now the only possibility is EU (for Europe), with US being the default if you don’t specify (or does ‘US’ work as a location?), but it doesn’t hurt to plan ahead.

I’ve only taken a cursory glance at the guts of S3 and S3Request so I’m not sure where you’d have to integrate this. If you were using PEAR/HTTP_Request, you’d use the addPutData method of HTTP_Request to add the appropriate snippet of XML to the put bucket request.

I’ll look into it more tomorrow and see what I can figure out, but I thought I’d see if you know offhand.

-- Heron on 11 June 2008

Firstly, awesome class – very useful :)

A little suggestion, if you wish, in the getBucket() function you could add the delimiter parameter as another argument, i’ve done this successfully for a project. Also in the initial checks within that function you have them all checking to see if $prefix !== ‘’ should this not be a check that each argument is not an empty string?
e.g. you have
if ($maxKeys !== null && $prefix !== ‘’)
should it not be
if ($maxKeys !== null && $maxKeys !== ‘’)

Thanks again for a great class.

-- Si on 13 June 2008

We have a client who’s phpBB forums receive a large number of hits constantly. The traffic isn’t so much of a concern, but the storage space is. Has anyone used this with the aforementioned software to store attachments? If so, we’d love to hear from you.

Karl

-- Mothership on 16 June 2008

@Karl: you’d have to modify phpBB to do that. It wouldn’t be very difficult, but it would take a bit of time (to track down everywhere that would need to be changed, if nothing else).

-- Heron on 16 June 2008

I’ve figured out how to specify Europe as the new bucket location, so I’ll tell you what I’ve added to make it work.

First I added an array, $extradata, as another private member variable of S3Request. Then I added a method addExtraData($data) similar in spirit to setHeader.

Next I added the following to the else statement in the ‘PUT’ case of the ‘request types’ switch in getResponse(), just after the CURLOPT_CUSTOMREQUEST line: [Sorry, the formatting doesn’t work well here.]

$strdata = ‘’;
foreach($extradata as $data)
{
$strdata .= $data;
}

if(strlen($strdata) > 0)
{
$temp = tmpfile();
fwrite($temp, $strdata);
curl_setopt($curl, CURLOPT_INFILE, $temp);
curl_setopt($curl, CURLOPT_INFILESIZE, strlen($strdata));
}

Finally, I added a third parameter to S3::putBucket, $europe, which defaults to false. Just before calling $rest->getResponse(), I do:

if($europe)
{
$data = ‘<CreateBucketConfiguration>\n’;
$data .= ‘ <LocationConstraint>EU</LocationConstraint>\n’;
$data .= ‘</CreateBucketConfiguration>’;
$rest->addExtraData($data);
}

Next I plan on seeing whether I can make listBuckets’ detailed return indicate whether a bucket is in Europe or not. I’ll let you know if/when I do.

Hope this helps.

-- Heron on 19 June 2008

@Heron: Excellent :) I’ve been meaning to add this in for some time now. There is an
easy way to do this with 0.2.x…

But to get a bucket location you can grab /bucket/uri?location – afaik there
isn’t a way to do it with listBuckets()…

-- Don on 19 June 2008

You are correct, there is unfortunately no way to get this information in listBuckets.

The changelog says the class supports the ?location and ?torrent parameters. How do I use them?

Specifically I’m trying to figure out the location of a bucket. The request looks like this, according to Amazon:

GET /?location HTTP/1.1
Host: quotes.s3.amazonaws.com
Date: Tue, 09 Oct 2007 20:26:04 +0000
Authorization: AWS 1ATXQ3HHA59CYF1CVS02:JUtd9kkJFjbKbkP9f6T/tAxozYY=

How do I get this set up with your s3 class? The getObjectInfo function doesn’t work, as it sends a HEAD request rather than a GET request as specified by Amazon. I think I’ll write another function to get this information until you have a moment to show me how to do it.

-- Heron on 20 June 2008

@Heron: I’ve added a getBucketLocation() method and a 3rd location parameter to putBucket().

But you’re right the getObjectInfo() HEAD request wouldn’t work for ?location.

-- Don on 20 June 2008

Awesome, that’s perfect.

Maybe next (if you’re open to suggestions or requests) you could add copy and rename methods for objects. Amazon lets you issue a COPY request to copy a file from one bucket+filename to another bucket+filename; that would make it easy for me to maintain European mirrors of my US data without having to upload the data twice.

You’d have to implement rename as a COPY (to the new file name, same bucket) and a DELETE (old file name).

-- Heron on 20 June 2008

@Heron: Added copyObject() :) Didn’t even know that was possible, thanks for the heads up!

-- Don on 20 June 2008

Awesome :) I’ll try it out at work tomorrow.

-- Heron on 20 June 2008

I used copyObject to implement a ‘rename’ function in my little interface, since I don’t actually need to be able to copy objects right now.

Works like a charm :)

-- Heron on 23 June 2008

Nice work. Works very well. You made my day! ;]

-- Ronny on 24 June 2008

Don,

I need to be able to modify the access permissions of an existing bucket. Something like $s3->modifyBucketPermissions($bucket, $permission) where $permission is public, public-read, or private. It might be useful to have a function to obtain the current permission settings for a given bucket as well. It’ll be far faster to wait for you to add this than to move the data elsewhere, recreate the bucket, and move the data back…

Here’s the documentation link for it:

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAccessPolicy.html

-- Heron on 24 June 2008

@Heron: You can do this:

$acp = $s3->getAccessControlPolicy($bucketName);

// Here you would modify $acp

// Then update it
$s3->setAccessControlPolicy($bucketName, ‘’, $acp);

-- Don on 25 June 2008

as has been said many times this is great. I have a very simple goal and am struggling to make it work. I want to get a secure file that is on my S3 and display it (gif, jpg, mov, whatever). I have successfully accessed the file but how do I display it now? I don’t need to actually DL the file just want to access it. Maybe this class is not good for that?

-- scott t on 25 June 2008

@scott: You probably don’t want to use the S3 class if you’re just trying to display the image on a webpage. Just link to it on the web page you want to display it on:

Perhaps what you’re trying to do is this:

Using the S3 class, grab a list of the contents of the bucket. Then, spit out an image tag (or a movie tag or whatever) for each item in the bucket, or for one particular item, etc. You don’t need to download the file using the S3 class though, just point a tag at it.

-- Heron on 26 June 2008

I keep getting this error for files over a certain limit. A file of 14KB is fine but a file of 140KB fails.

Warning: S3::putObject(): [505] Unexpected HTTP status in C:\Inetpub\wwwroot\s3\S3.php on line 325

-- Cameron on 5 July 2008

Hi,

This is great stuff. But can I use this library to list the contents of a particular bucket using a KEY? Like list all files in a buket “SWAP” which start with “SWAPNIL-” ?

I see you have a listBuckets() function which will list all buckets, but not the contents of each?

Please let me know as soon as you can.

Thanks,
Swapnil

-- Swapnil on 6 July 2008

@Cameron:

Are you uploading the file to your web server and then uploading it from there to S3, or are you uploading the file directly from your computer to S3?

@Swapnil:

$contents = $s3->getBucket($bucketName);

That will put the contents of $bucketName in $contents. You can then loop through $contents yourself and filter out the ones you don’t want. Read through example.php if you need to see how to use getBucket().

-- Heron on 6 July 2008

Hey, this is awesome.

Just wondering, is there a way to supress the error when a particular file/key doesn’t exist?

I’m trying to use this class to check if a particular file exists and if so, print out HTML code to display the thumbnail and link to the larger version.

Any ideas/suggestions?

Great script! B

-- Ben on 7 July 2008

@Ben: You might try the getObjectInfo() function. Pass it the bucket and uri; if it returns false, the object doesn’t exist.

E.g.:
if($s3->getObjectInfo($bucket, $objectname, false))
{ echo ‘anchor tag here’; }
else
{ echo ‘no such file’; }

The third parameter is important; you’ll get a bunch of object info instead of true/false if you don’t pass false for the third parameter.

-- Heron on 8 July 2008

Hello,
Delete operation doesn’t work whith files containing a space character.

-- johan on 9 July 2008

@johan: I ran into that same problem. You need to urlencode the object name in the delete request. For example:

$s3->deleteObject($bucketname, urlencode($objectname));

-- Heron on 9 July 2008

Same problem as Cameron #82. I used “S3 Firefox Organizer” to upload a bunch of test files to S3. I can use $s3->getBucket in a test script to obtain a list of the files and related data, all OK. But use of $s3->getObjectInfo on the same bucket and a specific object fails with “⁵⁰⁵ Unexpected HTTP status” error. The line in S3.php named is in the method code.

Thank you for this excellent module and I appreciate any help.

Chuck

-- Chuck on 10 July 2008

@Chuck: What is the name of the object you’re trying to get info on? Does it have a space or other symbols in the name? If so you might try using urlencode() on the object name before passing it to getObjectInfo().

-- Heron on 10 July 2008

Ah, you are the man! I used your idea to urlencode the filename and my test code now works.

Thank you very much for the prompt and effective response.

Chuck

-- Chuck on 10 July 2008

I’ve found the need to add some code to add Cache-Control header for my puts to S3. I think others would find this functionality useful as well. For my usage, I set the Cache-control so static files are not repeatedly downloaded from S3. Please let me know if/how you would like my code addition (e.g. diff, zip of s3.php or just the function).

Thanks for a great library!

-- Angelo Mandato on 12 July 2008

@Angelo: I’ve added some changes for request headers. Thanks for providing the info.

-- Don on 12 July 2008

I added the delimiter field in for getBucket():
<code>
public static function getBucket($bucket, $prefix = null, $marker = null, $maxKeys = null, $delimiter = ‘/’)

{

$rest = new S3Request(‘GET’, $bucket, ‘’);

if ($prefix !== null && $prefix !== ‘’) $rest->setParameter(‘prefix’, $prefix);

if ($marker !== null && $marker !== ‘’) $rest->setParameter(‘marker’, $marker);

if ($maxKeys !== null && $maxKeys !== ‘’) $rest->setParameter(‘max-keys’, $maxKeys);

if ($delimiter !== null && $delimiter !== ‘’) $rest->setParameter(‘delimiter’, $delimiter);

…
</code>

-- Christo on 13 July 2008

Just checked out the latest version (have never used this class before) and I’m getting an error of curl_init not defined. Any ideas?

-- Jed Wood on 14 July 2008

@Jed: You need the CURL extension.

-- Don on 14 July 2008

Looks like it does not work with file with space in its name.
It says:
Warning: S3::copyObject(mybucket1, blz21 – 22_t.jpg, mybucket2, blz21 – 22_t.jpg): [505] Unexpected HTTP status in /home/username/S3.php on line 446

Is it a bug or i have to encode the file name ?

-- Memphis on 18 July 2008

If you’d read previous posts, Memphis, you’d know the solution: You need to urlencode the object name before sending it.

-- Heron on 19 July 2008

Thanks, it works well. Do you know how to use getBucket() to list a part of objects in a bucket. For example I have 10000+ objects in my bucket and I want to list the first 1000 and then the next 1000 objects … in alphabetical order ? The $maxKeys param does not work for me in that case

-- Memphis on 19 July 2008

Memphis: If I remember right, it gives you the files from A-Z, 0-9, then a-z in that order. That’s not exactly alphabetical; it’s ordered by ASCII code. If you want it to be in case-insensitive alphabetic order, you’ll need to grab all the keys yourself, sort them, then cache them locally before displaying them to the user.

-- Heron on 20 July 2008

Is there any chance to add curl_multi functionality so that we can send requests in parallel?

-- Mike on 21 July 2008

@Mike: You could.. But unfortunately that functionality will not be added anytime soon.

-- Don on 21 July 2008

getObject method seems to be capable of writing object to a resource. but how can I make the process in which server side php requests S3 object with authentication and client user can download the object from browser?

-- Randy on 23 July 2008

Randy: I assume you’re asking because you’ve made your files private. You can generate a temporary download link using this method:

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?RESTAuthentication.html

Scroll down to the part labelled “Query String Request Authentication Alternative” near the bottom. Using this method, you can give your users a controlled way to download private objects from your buckets.

This thread goes in to more detail: http://developer.amazonwebservices.com/connect/thread.jspa?messageID=71402&#71402

-- Heron on 25 July 2008

Don, you forgot to update the version number in S3.php on SVN.

-- Heron on 28 July 2008

@Heron: Haha, I did, didn’t think anyone would notice :-)

-- Don on 28 July 2008

Hi, only small update to your S3.php class (in function inputFile), you can use
[ — cut — ]
if ( !is_readable($file) || !is_file($file) ) {
[ — cut — ]

instead of

[ — cut — ]
if (!file_exists($file) || !is_file($file) || !is_readable($file)) {
[ — cut — ]
“is_readable” will check if file really exists.
Cheers!

-- profes on 1 August 2008

Don,

I’m having trouble getting bucket logging to work. I have a private bucket “foobar.company.com” and another private bucket “company-access-logs”. It is my understanding that this:

$s3->setBucketLogging(‘foobar.company.com’, ‘company-access-logs’, ‘foobar-’);

should enable logging for bucket foobar.company.com into bucket company-access-logs. However, I get the following error:

Warning: S3::setBucketLogging(foobar.company.com, ): [InvalidTargetBucketForLogging] You must give the log-delivery group WRITE and READ_ACP permissions to the target bucket in /var/www/localhost/htdocs/s3/S3.php on line 491

Any idea what I’m doing wrong?

-- Heron on 4 August 2008

Solved via e-mail. Thanks, Don :)

-- Heron on 4 August 2008

Hi Don,
I modified your copyObject slightly to make a new setObjectInfo that allows for replacing meta data on objects without copying using x-amz-metadata-directive. This was very useful for me to fix Cache-Control headers on lots of images already uploaded. I’ve used it and it appears to work but one thing I noticed is that a new Content-Type needs to be supplied or else browsers may not respond as expected. I also added an acl parameter so that could be changed too. I added the code here but it doesn’t show well at all. Please email me and I’ll give it to you as it may be handy for others too. It was also nice for removing junk headers added by s3fox – didn’t like that stuff showing up anyway.

-- Chris Savery on 7 August 2008

A little help on creating folders.

Just add the folder name to the file name (not the temp file name).

So it would be like:
$s3->putObjectFile($fileTemp, $bucketName, ‘images/’.$fileName, S3::ACL_PUBLIC_READ, array(),‘image/jpg’);

Remember to add the contentType at the end if you want to be able to view your files via the browser.

-G

-- Giraldo on 8 August 2008

I had another “SignatureDoesNotMatch” issue this evening, might have been the same one affecting others. Some file uploads would work, some wouldn’t.

Turns out mime_content_type() was returning an extra tab character at the end of the string, which was throwing of the signature.

trim(mime_content_type($file)) on line 707 fixed the problem.

Thanks for a great bit of code!

-- David Kennedy on 13 August 2008

@David Kennedy: Thanks for the heads up!

-- Don on 13 August 2008

hi
i have some confusion with listBuckets() function
bcaz it shows the list of bucket which you had created but i have to validate new bucket name by matching all over the bucket names so how can i do that??
is there any reference or function for the same???

thanks in Advance

-- mahendra on 26 August 2008

Are you asking how to tell if a given bucket name is taken globally? You can’t, except by trying to register it. The best you can do is test it against the list of buckets you have registered yourself – that is, the ones returned by listBuckets.

-- Heron on 27 August 2008

hi Heron
Thanks 4 replying..
but actually when i use to create the bucket using s3 class as fuction putBucket ($bucketname) so what it does it just over right that bucket and updates the time of creation and no error received that bucket already exists or any error code

that’s why i am confused that why it is doing so either i am doing something wrong… or pl suggest me…

-- mahendra on 28 August 2008

Hey bro :)

Blog is coming along nicely. Loving the tutorials! About to code something up for my GuitarForum.co.za peeps using the S3 class. Thanks!

-- Norio on 31 August 2008

Your S3 PHP Class rocks!! Love that it does not require anything more than php 5.2. and I got my work done entotale in about 5hrs today. That time does include tweaks for human readable output. THANKS!

-- mmoney on 2 September 2008

@Don

Hey, nice script! Saved me there from alot of work. However I cant seem to find a solution to increase Filesize. It returns to page.php every time I try to upload a file. It loads for while but then goes back to the upload page. It occurs when I try to upload files greater than 10mb from HD to s3

-- Jones King on 2 September 2008

Jones King,

Is your webserver set to time out if the php script runs longer than some amount of time? You’ll probably want to check that…

- Heron

-- Heron on 3 September 2008

I’m currently trying to figure out the best way to initially upload the file to my server before using your class to send it over to my S3 bucket. The files are quite large (maximum 250MB) and a standard HTML POST form seems patch at best.

I’m using putObjectFile and passing it $_FILES[‘file’][‘tmp_name’]. I’ve heard that PHP loads the entire file into memory, or is that only if move_uploaded_file is called, am I mistaken entirely?

-- Nial Giacomelli on 3 September 2008

Nial,

I’m not sure what PHP’s behavior is; however paying for bandwidth twice (uploading to your server, then uploading to Amazon) seems superfluous. To help with this, Amazon provides a method to POST a file directly to your bucket, given your authorization, and then redirect to a page of your choice. You can find more information here:

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1434&ref=featured

Enjoy :)

-- Heron on 5 September 2008

@Nial: That’s a very good question… You are probably correct though. I would assume that the entire file would be loaded into memory on POST.

-- Don on 5 September 2008

Just wanted to say thanks for a great script! I’ve been using it to upload files from my webhost to amazon s3, but on some files I get this error:

Warning: S3::putObject(): [0] Missing input parameters in /directory/containing/S3.php on line 339

Anybody else have this problem? (I am using the 7/30/08 version)

-- Xenny on 6 September 2008

there’s a bug where sprintf blows up if there’s a % in the variable value in your sprintf calls. use %s placeholders everywhere and move the variables into the argument list

-- jasonmog on 8 September 2008

I have a lot of data to load into S3 so I did some benchmarking. I PUT 1000 small objects into a bucket with both classes:

CURL version took 7.5 minutes.

PEAR version took 4.5 minutes.

Of the 7.5 minutes, 6.5 are spent inside putObject’s $rest->getResponse() call.

I hope this will encourage you to try and optimize your class. I still plan to use it in production either way though so thanks for making it :-)

-- Brian on 10 September 2008

@Malone, @Don Regarding empty files, there is a bug in the S3.php code.

There are two conditional statements in S3::putObject that include $input[‘size’] > 0. If you replace those with $input[‘size’] > -1 it works fine.

I’m not sure why the check is there in the first place, but anyway this fix will still prevent negative sizes.

Cheers!

-- Chris Snyder on 10 September 2008

Found another bug, in that CURL is not verifying the SSL certificate coming from s3.amazonaws.com. If we care enough to use SSL, we should definitely be checking the certificate. ;-)

Anyway, issues have been added at code.google.com for this and comment #127.

Thanks for such a useful class!

-- Chris Snyder on 10 September 2008

@Chris: Regarding SSL verification: it is not a bug – CURL will spit out warnings otherwise.

-- Don on 11 September 2008

@Chris: The 0 byte check exists because I never expected people to upload empty files :) Will change in the next update anyway.

-- Don on 11 September 2008

@Brian: Sorry to say it but this isn’t a meaningful benchmark. Firstly, I cannot see how PHP code could be faster than C code. Secondly, the PEAR version buffers files in memory and using CURL we are actually able to have optimal unbuffered read/writes.

-- Don on 11 September 2008

Don – Brilliant class, implemented in no time at all!

Are you planning to add support for folders within buckets?

-- David on 11 September 2008

Hi Don, first of all, great work. I’m using your class to back up my database to S3. Quick question, what are the advantages or disadvantages of your class versus a PEAR based class? The class offered by neurofuzzy.net uses two PEAR extensions (HTTP_Request and HMAC). Is there a speed difference? Which class is better at buffering and streaming large files? Thanks for your help. Keep up the good work.

-- Kevin on 11 September 2008

@Kevin: There are a number of advantages – but the fact that there are no PEAR dependencies is usually enough for most. And if you are copying large files, this is the one you want to use.

-- Don on 11 September 2008

@David: There is a way to emulate folders by using “_$folder$” – but you would have to look into it… If you create a folder with S3Fox and retrieve a list using this class you will see how it is done.

-- Don on 11 September 2008

Hi Don, one more question. Are we still limited to 2GB files max? Thanks.

-- Kevin on 11 September 2008

@Kevin: Yes. See http://bugs.php.net/bug.php?id=27792 for more information.

-- Don on 11 September 2008

Don, when we add a delimiter to getBucket(), do you roll up the prefix? I notice the results still have all the prefixes attached to the front of the key names. Thanks.

-- Kevin on 13 September 2008

Hmm, this is not working for me. Tried it in various hosting environments and I’m still experiencing the issues outlined in comment #14:

“When I try to upload large files it will just stop after a couple minutes. It gives no error.”

Any ideas?

-- Sven on 14 September 2008

@Sven I’m having the same problem but only on large files. I tried setting CURLOPT_TIMEOUT to a high number to no avail.

-- Bryan on 17 September 2008

@Sven and Bryan: If you’re using this class to upload from a user’s computer to an S3 bucket, you may want to consider POSTing directly to the S3 bucket, as explained in this article:

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1434&ref=featured

If, however, you’re using it to upload files from your server to S3, you’ll need to figure out what your Apache settings are and tweak them so Apache doesn’t time out.

-- Heron on 17 September 2008

@141 – The following PHP specific tutorial is good, and uses AJAX to upload files

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1618

-- David on 23 September 2008

I have a question similiar to Post 103. I have a file that is private and needs to be automatically downloaded. How is this be done with the getObject? Or is the best way to use “Query String Request Authentication Alternative?” Thanks.

-- Sam on 25 September 2008

Sam: You need the user’s browser to be able to download it? You’ll want to generate a temporary URL giving permission to download the private file, as in this article:

http://developer.amazonwebservices.com/connect/thread.jspa?messageID=71402&#71402

-- Heron on 25 September 2008

I’ve got following problem – sometimes (10% probability?) when uploading to S3, my php (CLI, 5.2.0-8+etch11, libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5) hangs there forever eating 100% cpu. Strace shows only those two lines again and again…

poll([{fd=4, events=POLLOUT, revents=POLLHUP}], 1, 0) = 1
poll([{fd=4, events=POLLOUT, revents=POLLHUP}], 1, 1000) = 1

It happens in putObjectFile() uploading cca 750MB file.

Any idea what the problem might be?

-- marek on 26 September 2008

@Heron: Thanks for your help.

-- Sam on 26 September 2008

Your class didn’t allow for authenticating via query string:

$authentication_params = “AWSAccessKeyId=” . self::$__accessKey; $authentication_params.= “&Expires=$expires”; $authentication_params.= “&Signature=$signature”;

$link = “http://$bucket.s3.amazonaws.com/$resource?$authentication_params”;

return $link; //url encode this? //echo “<a href=\”“.htmlentities($link).”\”>Authenticated Link</a>”;

}

-- pixelterra on 3 October 2008

Recent Posts

Links

Blog

Amazon S3 PHP Class

Comments