Server-side file upload progress with Symfony 1.4
I researched many methods of creating a server-side upload progress bar for file upload, that would work across all browsers. Also, because my development setup was different from the live environment, I wanted something I could use with multiple solutions. This post covers the following options:
APC's apc_fetch (Apache + PHP5 + APC)
PHP 5.4's session.upload_progress (Apache + PHP5.4+)
Nginx's HttpUpload and HttpUploadProgress modules (Nginx + PHP-FPM/ PHP-CGI)
If you are using Nginx with PHP-FPM or PHP-CGI and want a server-side solution, you'll have to use option 3 as the file uploads are buffered - thus, using options 1 or 2 would result in the code returning false until the file upload was complete, and then returning 100%.
Note: Ensure upload_max_filesize and post_max_size are set to appropriate values in the php.ini.
Option 1 - apc_fetch:
Options for APC should be set in the php.ini or apc.ini file (depending on your setup). The apc.max_file_size should not exceed those values set in for upload_max_filesize and post_max_size.
apc.rfc1867 = On apc.max_file_size = 100M apc.rfc1867_freq = 10k
Option 2:
Options for PHP5.4+ session.upload_progress should be set in the php.ini file (they should be there by default for new installations of PHP5.4).
; Enable upload progress tracking in $_SESSION session.upload_progress.enabled = On ; Cleanup the progress information as soon as all POST data has been read session.upload_progress.cleanup = Off ; A prefix used for the upload progress key in $_SESSION session.upload_progress.prefix = "upload_progress_" ; The index name (concatenated with the prefix) in $_SESSION session.upload_progress.name = "PHP_SESSION_UPLOAD_PROGRESS" ; How frequently the upload progress should be updated. session.upload_progress.freq = "1%" ; The minimum delay between updates, in seconds session.upload_progress.min_freq = "1"
Option 3:
I installed the nginx-extras package for Ubuntu 12.04 which includes the appropriate third party modules. If this is not available then you will have to manually compile nginx with the HttpUpload and HttpUploadProgress third party modules.
In nginx.conf (before including sites-enabled etc) inside the http {} directive you need to enable upload_progress tracking.
# upload tracking - allocate memory upload_progress uploads 5m;
In the vhost (after the 'location /' rule, inside the server {} directive) set up upload and upload_progress for the specific site. Note: client_max_body_size should not exceed that set in the php.ini.
# for upload management client_max_body_size 100m; location @controller { rewrite ^(.*) /index.php$1 last; } # to mimic the symfony route location /upload-progress { upload_progress_json_output; # return pure JSON report_uploads uploads; } location /submit-upload { # pass altered request - update action to deal with no $_FILES array (only accepts $_POST) upload_pass @controller; # Store files to this directory (update to match server) # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist upload_store /home/username/tmp 1; # Set specified fields in request body (match $_FILES fields) upload_set_form_field "$upload_field_name[name]" "$upload_file_name"; upload_set_form_field "$upload_field_name[type]" "$upload_content_type"; upload_set_form_field "$upload_field_name[tmp_name]" "$upload_tmp_path"; upload_aggregate_form_field "$upload_field_name[size]" "$upload_file_size"; # ensure other fields are passed through (obviously this should match your form - can be regex) upload_pass_form_field "entry"; # clean up thing upload_cleanup 400 404 499 500-505; # track uploads, remember connections for 30s track_uploads uploads 30s; }
The PHP (Symfony 1.4)
Set up your site and schema as normal.
In routing.yml
submit: # the form url: /submit param: { module: index, action: submit } submit_upload: # the form action url: /submit-upload param: { module: index, action: submitUpload } upload_progress: # emulating the nginx progress module return url: /upload-progress param: { module: index, action: uploadProgress }
In the form template (using a symfony form for the Entry model with a field of 'video' for the file upload):
NOTE: The hidden input containing the progress key MUST be before the file input for these to work.
<?php $progress_key = rand(); ?> <?php echo $form->renderFormTag(url_for('@submit_upload?X-Progress-ID='.$progress_key), array('id'=>'the-form')); ?> ... <input type="hidden" name="MAX_FILE_SIZE" value="<?php echo sfConfig::get('app_video_max_file_size'); ?>" /> <input type="hidden" name="<?php echo ini_get('apc.rfc1867_name') || ini_get('session.upload_progress.name') // depending on option 1/2; ?>" id="progress_key" value="<?php echo $progress_key; ?>" /> <?php echo $form['video']->renderRow(); ?> <div id="progress_bar"></div> ... </form>
For options 1 and 2 (apc_fetch and session.upload_progress) I chose to mimic the HttpUploadProgress JSON return so that the code was interchangeable across environments. In the action:
/** * Executes upload progress action * Mimics return of NginxHttpUploadProgressModule using apc_fetch or session.upload_progress (PHP5.4+) on Apache - delete sections as appropriate * * @see http://wiki.nginx.org/NginxHttpUploadProgressModule * @param sfRequest $request A request object */ public function executeUploadProgress(sfWebRequest $request) { $this->setLayout(false); $return = array('state' => 'starting'); if ($request->hasParameter('X-Progress-ID')) { // if apc_fetch $key = ini_get('apc.rfc1867_prefix') . $request->getParameter('X-Progress-ID'); $status = apc_fetch($key); // if session.upload_progress $key = ini_get('session.upload_progress.prefix') . $request->getParameter('X-Progress-ID'); $status = $_SESSION[$key]; if ($status) { // if apc_fetch $current = 'current'; $total = 'total'; // if session.upload_progress $current = 'bytes_processed'; $total = 'content_length'; if ($status[$current] == $status[$total]) $return = array('state' => 'done'); else $return = array('state' => 'uploading', 'received' => $status[$current], 'size' => $status[$total]); } } $this->getResponse()->setContentType('application/json'); return $this->renderText(json_encode($return)); }
And finally the process submit action (for all options):
/** * Process the form submission * Will work with all options for file upload progress - delete sections as appropriate * * @param sfWebRequest $request */ public function executeSubmitUpload(sfWebRequest $request) { $this->setTemplate('submit'); // share template with submit, in case of errors $this->form = new EntryForm(new Entry()); if (($request->isMethod(sfWebRequest::POST) || $request->isMethod(sfWebRequest::PUT)) && $request->hasParameter('entry')) { $parameters = $request->getParameter('entry'); // if using apc_fetch or session.upload_progress - have $_POST with $_FILES $files = $request->getFiles('entry'); // if using nginx upload progress // uploads file to temporary directory specified in vhost and then adds specified fields to $_POST (I mimicked the $_FILES array) if (isset($parameters['video'])) { $files = array('video' => $parameters['video']); $files['video']['error'] = 0; // @TODO: Unsure how this handles errors unset($parameters['video']); } else $files = array('video' => array()); $this->form->bind($parameters, $files); if ($this->form->isValid()) { $this->entry = $this->form->save(); // if using nginx upload - delete temp file if (isset($files['video']) && isset($files['video']['tmp_name'])) { if (is_file($files['video']['tmp_name'])) unlink($files['video']['tmp_name']); } // thankyou page etc. } } }
Note: I haven't dealt with the situation where nginx returns an error on file upload.
The Javascript
And the javascript (using jQuery 1.9 with the migration script for $.browser support). Create this in a separate file for use in the webkit iframe solution. Obviously any ids and paths will need to be updated to match your setup, along with how the progress bar is rendered.
var UPLOAD = { interval : null, // call in document ready init: function() { $.migrateMute = true; UPLOAD.submit(); if ($('#the-form').length > 0) { // Uses jquery migration for browser detection as 1.9 removes it, and deprecated from 1.3 // Reason can't use anything else is because it's not a feature detect per-se, not covered by // modernizr - I just need to know if webkit - as this doesn't support simultaneous POST + AJAX if ($.browser.webkit) { // Create an iframe to use var iframe = document.createElement('iframe'); iframe.name = iframe.id = 'progressFrame'; $(iframe).css({ width: '0', height: '0', position: 'absolute', top: '-3000px', left: '-3000px' }); document.body.appendChild(iframe); var d = iframe.contentWindow.document; d.open(); d.write(''); d.close(); // Add scripts for us to call var b = d.head; var s = d.createElement('script'); s.src = "//ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"; /* must be sure that jquery is loaded */ s.onload = function() { var s1 = d.createElement('script'); s1.src = "//code.jquery.com/jquery-migrate-1.0.0.js"; b.appendChild(s1); var s2 = d.createElement('script'); s2.src = "/js/upload.js"; b.appendChild(s2); } b.appendChild(s); } } }, submit: function() { $(document).on('submit', '#the-form', function(event){ var uploadProgress = ($.browser.webkit ? progressFrame.UPLOAD.progress : UPLOAD.progress); UPLOAD.interval = setInterval(uploadProgress, 1000); // every second }); }, progress: function() { $.migrateMute = true; var progress_key = ($.browser.webkit ? $('#progress_key', parent.document).val() : $('#progress_key').val()); var uploadProgressUrl = ($.browser.webkit ? parent.Settings.uploadProgress : Settings.uploadProgress); // create new AJAX request that returns response object $.ajax({ type: 'GET', dataType: 'json', url: uploadProgressUrl + '?X-Progress-ID=' + progress_key + '&rand=' + Math.floor(Math.random() * 1000), success : function(data) { var bar = ($.browser.webkit ? $('#progress_bar', parent.document) : $('#progress_bar')); if (data.state == 'uploading') { var percent = Math.round((data.received / data.size) * 100); bar.html(percent + '%'); if (percent >= 100) { clearInterval(UPLOAD.interval); // Upload done, stop polling. } } else if (data.state == 'starting') { bar.html('0%'); } else if (data.state == 'done') { clearInterval(UPLOAD.interval); // Upload done, stop polling. bar.html('100%'); } } }); } };
Additional resources used for research and to compile the code:
http://serverfault.com/questions/420821/nginx-buffering-data-before-sending-to-fastcgi
http://blog.martinfjordvald.com/2010/08/file-uploading-with-php-and-nginx/
http://stackoverflow.com/questions/952267/safari-doesnt-allow-ajax-requests-after-form-submit
https://github.com/drogus/jquery-upload-progress
And because Ubuntu 12.04 does not come with PHP5.4 (Suhosin issues) - http://www.zimbio.com/Ubuntu+Linux/articles/D_AsJR2qAL6/How+Upgrade+PHP+5+4+Ubuntu.
Obviously there are other solutions, such as the PECL Upload progress module (Apache only), but they are not covered in this post.