使用boto将文件从ec2传输到s3时出错

前端之家收集整理的这篇文章主要介绍了使用boto将文件从ec2传输到s3时出错前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

我按照这个程序link将我的mongodump上传到s3.

bash脚本

  1. #!/bin/sh
  2. MONGODB_SHELL='/usr/bin/mongo'
  3. DUMP_UTILITY='/usr/bin/mongodump'
  4. DB_NAME='amicus'
  5. date_now=`date +%Y_%m_%d_%H_%M_%S`
  6. dir_name='db_backup_'${date_now}
  7. file_name='db_backup_'${date_now}'.bz2'
  8. log() {
  9. echo $1
  10. }
  11. do_cleanup(){
  12. rm -rf db_backup_2010*
  13. log 'cleaning up....'
  14. }
  15. do_backup(){
  16. log 'snapshotting the db and creating archive' && \
  17. ${MONGODB_SHELL} admin fsync_lock.js && \
  18. ${DUMP_UTILITY} -d ${DB_NAME} -o ${dir_name} && tar -jcf $file_name ${dir_name}
  19. ${MONGODB_SHELL} admin unlock.js && \
  20. log 'data backd up and created snapshot'
  21. }
  22. save_in_s3(){
  23. log 'saving the backup archive in amazon S3' && \
  24. python aws_s3.py set ${file_name} && \
  25. log 'data backup saved in amazon s3'
  26. }
  27. do_backup && save_in_s3 && do_cleanup

aws_s3.py

  1. ACCESS_KEY=''
  2. SECRET=''
  3. BUCKET_NAME='s3:///s3.amazonaws.com/database-backup' #note that you need to create this bucket first
  4. from boto.s3.connection import S3Connection
  5. from boto.s3.key import Key
  6. def save_file_in_s3(filename):
  7. conn = S3Connection(ACCESS_KEY,SECRET)
  8. bucket = conn.get_bucket(BUCKET_NAME)
  9. k = Key(bucket)
  10. k.key = filename
  11. k.set_contents_from_filename(filename)
  12. def get_file_from_s3(filename):
  13. conn = S3Connection(ACCESS_KEY,SECRET)
  14. bucket = conn.get_bucket(BUCKET_NAME)
  15. k = Key(bucket)
  16. k.key = filename
  17. k.get_contents_to_filename(filename)
  18. def list_backup_in_s3():
  19. conn = S3Connection(ACCESS_KEY,SECRET)
  20. bucket = conn.get_bucket(BUCKET_NAME)
  21. for i,key in enumerate(bucket.get_all_keys()):
  22. print "[%s] %s" % (i,key.name)
  23. def delete_all_backups():
  24. #FIXME: validate filename exists
  25. conn = S3Connection(ACCESS_KEY,key in enumerate(bucket.get_all_keys()):
  26. print "deleting %s" % (key.name)
  27. key.delete()
  28. if __name__ == '__main__':
  29. import sys
  30. if len(sys.argv) < 3:
  31. print 'Usage: %s

但不断收到此错误

  1. Traceback (most recent call last):
  2. File "aws_s3.py",line 42,in

做了一点我的研究,发现它是boto中的某种bug.如何继续这个?

最佳答案
由于我没有得到任何更新如何使其工作,我在我的bash脚本中使用了s3cmd.但我还是要测试文件> 1gb.

这是更新的代码

  1. #!/bin/sh
  2. MONGODB_SHELL='/usr/bin/mongo'
  3. DUMP_UTILITY='/usr/bin/mongodump'
  4. DB_NAME='amicus'
  5. date_now=`date +%Y_%m_%d_%H_%M_%S`
  6. dir_name='db_backup_'${date_now}
  7. file_name='db_backup_'${date_now}'.bz2'
  8. log() {
  9. echo $1
  10. }
  11. do_cleanup(){
  12. rm -rf db_backup_2010*
  13. log 'cleaning up....'
  14. }
  15. do_backup(){
  16. log 'snapshotting the db and creating archive' && \
  17. ${DUMP_UTILITY} -d ${DB_NAME} -o ${dir_name} && tar -jcf $file_name ${dir_name}
  18. log 'data backd up and created snapshot'
  19. }
  20. save_in_s3(){
  21. log 'saving the backup archive in amazon S3' && \
  22. python aws_s3.py set ${file_name} && \
  23. s3cmd put ${file_name} s3://YOURBUCKETNAME
  24. log 'data backup saved in amazon s3'
  25. }
  26. do_backup && save_in_s3 && do_cleanup

猜你在找的Python相关文章