Bug #3826
closed
curl command for download a file fails
Added by Fabrice FLORE-THÉBAULT over 11 years ago.
Updated about 11 years ago.
Description
The curl command actually generated to download the file fails.
Downloading the file with wget is correct.
- rudder-agent-2.6.3.release-1.EL.5, Rudder v.2.6.3
Directive "Download a file"¶
- Absolute path to download file to on a node:
/root/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
Execution of the directive¶
- /opt/rudder/bin/cf-agent -KI | grep bundle.tar
-> Executing '/usr/bin/curl -f --proxy '' -o "/root/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar/from/http://cdn.mysql.com/' ... (no timeout)
I: Last 3 quoted lines were generated by promiser "/usr/bin/curl -f --proxy '' -o "/root/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar/from/http://cdn.mysql.com/"
-> Completed execution of /usr/bin/curl -f --proxy '' -o "/root/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar/from/http://cdn.mysql.com/
R: @@downloadFile@@result_repaired@@d631b6be-d007-4db5-a04d-22527903bc92@@23a33c6c-afc4-4bd0-b37c-f97823c5986a@@2@@Download a file@@http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar/from/http://cdn.mysql.com/@@2013-08-08 14:25:02+02:00##59b4508b-95ae-4ced-9bb3-2ebab3f39362@#The file http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar/from/http://cdn.mysql.com/ has been successfully downloaded
- ls -alh MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
8,0K -rwxr----- 1 root root 200 aoû 8 14:25 MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
the curl command doesn't work as expected¶
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 200 100 200 0 0 544 0 --:--:-- --:--:-- --:--:-- 0
with wget it is working as expected¶
--2013-08-08 14:27:59-- http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar/from/http://cdn.mysql.com/
Résolution de dev.mysql.com... 137.254.60.11
Connexion vers dev.mysql.com|137.254.60.11|:80...connecté.
requête HTTP transmise, en attente de la réponse...302 Found
Emplacement: http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar [suivant]
--2013-08-08 14:27:59-- http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
Résolution de cdn.mysql.com... 2.21.243.50, 23.62.99.17
Connexion vers cdn.mysql.com|2.21.243.50|:80...connecté.
requête HTTP transmise, en attente de la réponse...200 OK
Longueur: 311285760 (297M) [application/x-tar]
Saving to: `MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar.1'
- Assignee set to Matthieu CERDA
- Priority changed from N/A to 2
- Target version changed from 2.6.4 to 2.4.8
This is a quit annoiyign bug about that Technique.
It should happen in Rudder 2.4 too, and will be fixed in that version.
Mathieu or Nicolas can you look into this ?
- Target version changed from 2.4.8 to 2.4.9
- Status changed from New to Discussion
- Assignee changed from Matthieu CERDA to Nicolas PERRON
I just tested with the updated URL:
/usr/bin/curl -f --proxy '' -o "/root/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
There is no problem, the file gets downloaded correctly.
I think that the MySQL site did funky things when dealing when some user agents like the "curl" one ...
Is there still a problem nowadays with that ?
NPE: Can you confirm that on your side it works too ?
- Assignee changed from Nicolas PERRON to Matthieu CERDA
Matthieu CERDA wrote:
I just tested with the updated URL:
[...]
There is no problem, the file gets downloaded correctly.
I think that the MySQL site did funky things when dealing when some user agents like the "curl" one ...
Is there still a problem nowadays with that ?
NPE: Can you confirm that on your side it works too ?
The curl command works for me.... but the problem could be that it takes too many time. This is a file of 296M !
Le 20/09/2013 16:11, redmine@rudder-project.org a écrit :
Issue #3826 has been updated by Nicolas PERRON.
- Assignee changed from /Nicolas PERRON/ to /Matthieu CERDA/
NPE: Can you confirm that on your side it works too ?
The curl command works for me.... but the problem could be that it
takes too many time. This is a file of 296M !
It takes 18s on a machine in the Online datacenter, and approx. 5
minutes on a Free ADSL. Should not be a problem :)
- Status changed from Discussion to In progress
- Assignee changed from Matthieu CERDA to Nicolas PERRON
I'm testing this on an Orchestrator.
- Status changed from In progress to Rejected
- Assignee changed from Nicolas PERRON to Matthieu CERDA
- % Done changed from 0 to 100
I confirm a Directive on Rudder 2.6 downlading "http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" works correctly:
R: @@Common@@log_info@@hasPolicyServer-root@@common-root@@5@@common@@StartRun@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Start execution
R: @@Common@@log_repaired@@hasPolicyServer-root@@common-root@@5@@Update@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Node's policy (CFEngine promises) updated
R: @@Common@@result_repaired@@hasPolicyServer-root@@common-root@@5@@Update@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Policy or dependencies were updated or CFEngine service restarted
!! Method invoked repairs
R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@5@@Security parameters@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#The internal environment security is acceptable
R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@5@@Red Button@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Red Button is not in effect, continuing as normal...
R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@5@@Process checking@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#There is an acceptable number of CFEngine processes running on the machine
R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@5@@CRON Daemon@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#The CRON daemon is running
-> Deleted file /var/rudder/cfengine-community/outputs/cf_debian_7_64_labo_normation_com__1377099387_Wed_Aug_21_17_36_27_2013_0x7f6177b6d700
R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@5@@Binaries update@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#The CFengine binaries in /var/rudder/cfengine-community/bin are up to date
R: @@Common@@log_info@@hasPolicyServer-root@@common-root@@5@@Log system for reports@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Detected running syslog as rsyslog
R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@5@@Log system for reports@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Logging system for report centralization is already correctly configured
-> Executing '/usr/bin/curl -s -f --proxy '' -o "/var/rudder/tmp/uuid.txt" http://192.168.110.42/uuid' ... (no timeout)
-> Completed execution of /usr/bin/curl -s -f --proxy '' -o "/var/rudder/tmp/uuid.txt" http://192.168.110.42/uuid
R: @@Inventory@@log_info@@inventory-all@@inventory-all@@2@@inventory@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#An inventory was already sent less than 8 hours ago
R: @@Inventory@@result_success@@inventory-all@@inventory-all@@2@@inventory@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#Next inventory scheduled between 00:00 and 06:00
-> Executing '/usr/bin/curl -f --proxy '' -o "/tmp/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar' ... (no timeout)
32 296M 32 95.5M 0 0 1925k 0 0:02:36 0:00:49 0:01:47 989k
: "...r/bin/curl -f -": % Total % Received % Xferd Average Speed Time Time Time Current
Q: "...r/bin/curl -f -": Dload Upload Total Spent Left Speed
I: Last 3 quoted lines were generated by promiser "/usr/bin/curl -f --proxy '' -o "/tmp/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar"
-> Completed execution of /usr/bin/curl -f --proxy '' -o "/tmp/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar" http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
R: @@downloadFile@@result_repaired@@e8ab81b9-0e8f-43e3-9ea6-5cb4f0b87713@@a9b0a947-13df-4188-90b3-70997c11b05b@@1@@Download a file@@http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#The file http://cdn.mysql.com/Downloads/MySQL-5.6/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar has been successfully downloaded
R: @@MOTD@@result_success@@70ca49a2-7696-470e-b785-452230aabd78@@6a8f28c3-1bbb-4580-bc1f-e512dcb72ca1@@6@@MOTD Configuration@@None@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#The MOTD file was correct
!! Method invoked repairs
!! Method invoked repairs
!! Method invoked repairs
R: @@Common@@log_info@@hasPolicyServer-root@@common-root@@5@@common@@EndRun@@2013-09-20 17:38:20+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#End execution
root@debian-7-64:/tmp# ls -hal MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
-rwxr--r-- 1 root root 297M sept. 20 17:41 MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
root@debian-7-64:/tmp# tar tvf /tmp/MySQL-5.6.13-1.rhel5.x86_64.rpm-bundle.tar
-rw-r--r-- bteam/staff 4560042 2013-07-12 14:04 MySQL-devel-5.6.13-1.rhel5.x86_64.rpm
-rw-r--r-- bteam/staff 5108409 2013-07-12 14:06 MySQL-shared-compat-5.6.13-1.rhel5.x86_64.rpm
-rw-r--r-- bteam/staff 113487964 2013-07-12 14:04 MySQL-embedded-5.6.13-1.rhel5.x86_64.rpm
-rw-r--r-- bteam/staff 89692969 2013-07-12 14:05 MySQL-server-5.6.13-1.rhel5.x86_64.rpm
-rw-r--r-- bteam/staff 2392023 2013-07-12 14:06 MySQL-shared-5.6.13-1.rhel5.x86_64.rpm
-rw-r--r-- bteam/staff 73082091 2013-07-12 14:06 MySQL-test-5.6.13-1.rhel5.x86_64.rpm
-rw-r--r-- bteam/staff 22953357 2013-07-12 14:04 MySQL-client-5.6.13-1.rhel5.x86_64.rpm
Rejecting this.
Also available in: Atom
PDF
Updated by 
Updated by Nicolas PERRON 
Updated by Matthieu CERDA