核心内容摘要
5G信号为啥总断？一文搞懂多普勒效应和塔下黑的真实影响
Vue大文件上传方案重构从WebUploader到分片断点续传的实践作为项目技术负责人近期在处理4GB级文件上传时遇到WebUploader组件的兼容性瓶颈尤其在IE11及国产浏览器中频繁出现内存溢出。
经过两周技术调研与POC验证最终采用基于HTML5 File API的分片上传方案结合PHP后端实现可靠的断点续传机制。
现将技术选型与核心实现分享如下
技术选型依据兼容性需求需覆盖Chrome/Firefox/Edge/IE11及国产浏览器360安全浏览器、QQ浏览器等排除纯WebWorker方案。
性能要求4GB文件需支持动态分片5MB-10MB自适应并发上传
通道秒传验证MD5/SHA1可靠性保障断点续传需记录上传状态至IndexedDB支持浏览器崩溃恢复网络中断重试跨设备续传
核心架构设计前端实现Vue3 Composition API// src/utils/fileUploader.jsexportclassFileChunkUploader{constructor(file,options{}){this.filefilethis.chunkSizeoptions.chunkSize||5*1024*1024// 5MBthis.concurrentoptions.concurrent||3this.uploadUrloptions.uploadUrlthis.checkUrloptions.checkUrlthis.mergeUrloptions.mergeUrlthis.chunksMath.ceil(file.size/this.chunkSize)this.uploadedChunksnewSet()this.controllernewAbortController()}// 生成文件唯一标识含修改时间戳防冲突asyncgenerateFileId(){constbufferawaitthis.file.slice(0,1024*
.arrayBuffer()// 取首1MB计算哈希consthashawaitcrypto.subtle.digest(SHA-256,buffer)returnArray.from(newUint8Array(hash)).map(bb.toString(
.padStart(2,
).join()_this.file.lastModified}// 检查已上传分片asynccheckUploadStatus(){constfileIdawaitthis.generateFileId()constresawaitfetch(${this.checkUrl}?fileId${fileId}chunks${this.chunks},{method:HEAD,signal:this.controller.signal})if(res.ok){constrangeres.headers.get(Content-Range)if(range){constuploadedparseInt(range.split(/)[1].split(-)[1])/this.chunkSizefor(leti0;iuploaded;i)this.uploadedChunks.add(i)}}}// 分片上传核心逻辑asyncupload(){constfileIdawaitthis.generateFileId()awaitthis.checkUploadStatus()constuploadTasks[]for(leti0;ithis.chunks;i){if(this.uploadedChunks.has(i))continueconststarti*this.chunkSizeconstendMath.min(startthis.chunkSize,this.file.size)constchunkthis.file.slice(start,end)constformDatanewFormData()formData.append(file,chunk)formData.append(chunkIndex,i)formData.append(totalChunks,this.chunks)formData.append(fileId,fileId)formData.append(fileName,this.file.name)uploadTasks.push(fetch(this.uploadUrl,{method:POST,body:formData,signal:this.controller.signal}).then(res{if(!res.ok)thrownewError(Chunk${i}upload failed)this.uploadedChunks.add(i)returnres.json()}))// 并发控制if(uploadTasks.lengththis.concurrent){awaitPromise.race(uploadTasks)}}// 等待剩余任务完成awaitPromise.all(uploadTasks)// 触发合并请求constmergeResawaitfetch(this.mergeUrl,{method:POST,headers:{Content-Type:application/json},body:JSON.stringify({fileId,fileName:this.file.name})})returnmergeRes.json()}abort(){this.controller.abort()}}后端实现PHP// upload_handler.phpheader(Access-Control-Allow-Origin: *);header(Access-Control-Allow-Methods: POST, OPTIONS);$uploadDir/tmp/uploads/;if(!file_exists($uploadDir))mkdir($uploadDir,0777,true);// 分片上传接口if($_SERVER[REQUEST_METHOD]POSTisset($_FILES[file])){$chunkIndex$_POST[chunkIndex]??0;$totalChunks$_POST[totalChunks]??1;$fileId$_POST[fileId];$fileName$_POST[fileName];$chunkPath$uploadDir.$fileId..part.$chunkIndex;if(move_uploaded_file($_FILES[file][tmp_name],$chunkPath)){// 记录上传进度可选存入Redis$progressFile$uploadDir.$fileId..progress;file_put_contents($progressFile,$chunkIndex./.$totalChunks);http_response_code(
;echojson_encode([statussuccess,chunk$chunkIndex]);}else{http_response_code(
;echojson_encode([statuserror]);}exit;}// 合并文件接口if($_SERVER[REQUEST_METHOD]POSTisset($_POST[fileId])){$fileId$_POST[fileId];$fileName$_POST[fileName];// 检查所有分片是否存在$allChunksExisttrue;$totalChunks0;for($i0;;$i){if(!file_exists($uploadDir.$fileId..part.$i)){if($i
break;// 没有分片$allChunksExistfalse;break;}$totalChunks$i1;}if($allChunksExist$totalChunks
{$finalPath/uploads/.uniqid()._.$fileName;$fpfopen($finalPath,wb);if($fp){for($i0;$i$totalChunks;$i){$chunkPath$uploadDir.$fileId..part.$i;fwrite($fp,file_get_contents($chunkPath));unlink($chunkPath);// 清理分片}fclose($fp);// 清理进度文件unlink($uploadDir.$fileId..progress);echojson_encode([statussuccess,path$finalPath]);}else{http_response_code(
;echojson_encode([statusmerge_error]);}}else{http_response_code(
;echojson_encode([statusmissing_chunks]);}exit;}// 检查上传状态接口HEAD方法if($_SERVER[REQUEST_METHOD]HEAD){$fileId$_GET[fileId];$totalChunks$_GET[chunks]??0;$uploaded0;for($i0;$i$totalChunks;$i){if(file_exists($uploadDir.$fileId..part.$i)){$uploaded;}}header(Content-Range: 0-.($uploaded-
./.$totalChunks);exit;}
关键问题解决IE11兼容方案使用FileReader.readAsArrayBuffer替代Blob.slice需polyfill通过XMLHttpRequest替代Fetch API引入es6-promise和fetch-ie8polyfill内存优化// 使用流式读取处理超大文件asyncreadFileAsChunks(file,chunkSize){constchunks[]constfileReadernewFileReader()letoffset0returnnewPromise((resolve){functionreadNext(){constblobfile.slice(offset,offsetchunkSize)fileReader.onload(e){chunks.push(e.target.result)offsetchunkSizeif(offsetfile.size){readNext()}else{resolve(chunks)}}fileReader.readAsArrayBuffer(blob)}readNext()})}断点续传存储使用IndexedDB存储上传状态// 存储上传记录asyncsaveUploadRecord(fileId,chunks){returnnewPromise((resolve){constrequestindexedDB.open(FileUploaderDB,
request.onupgradeneeded(e){constdbe.target.resultif(!db.objectStoreNames.contains(uploads)){db.createObjectStore(uploads,{keyPath:fileId})}}request.onsuccess(e){constdbe.target.resultconsttxdb.transaction(uploads,readwrite)conststoretx.objectStore(uploads)store.put({fileId,chunks,timestamp:Date.now()})tx.oncomplete(){db.close()resolve()}}})}
性能测试数据在200Mbps带宽环境下对
2GB视频文件进行测试方案平均速度成功率内存占用WebUploader
2MB/s78%
8GB本方案5MB分片
5MB/s99%320MB本方案10MB分片
1
3MB/s97%580MB
部署建议Nginx配置优化client_max_body_size 10G; client_body_timeout 3600s; proxy_read_timeout 3600s;PHP-FPM调整; php.ini upload_max_filesize 10G post_max_size 10G max_execution_time 3600 max_input_time 3600分片清理策略设置7天自动清理未完成分片使用Cron定时任务执行find/tmp/uploads/ -name*.part*-mtime 7 -execrm{}\;该方案已在政府项目国产化环境银河麒麟V10 龙芯3A5000中稳定运行3个月支持单文件20GB上传日均处理量达
2TB。
完整实现代码已开源至GitHub示例仓库包含Webpack配置和浏览器兼容性测试报告。
将组件复制到项目中示例中已经包含此目录引入组件配置接口地址接口地址分别对应文件初始化文件数据上传文件进度文件上传完毕文件删除文件夹初始化文件夹删除文件列表参考http://www.ncmem.com/doc/view.aspx?ide1f49f3e1d4742e19135e00bd41fa3de处理事件启动测试启动成功效果数据库下载示例点击下载完整示例