{"id":47359,"date":"2019-12-07T03:01:30","date_gmt":"2019-12-06T17:01:30","guid":{"rendered":"http:\/\/www.rjmprogramming.com.au\/ITblog\/?p=47359"},"modified":"2020-04-20T08:13:48","modified_gmt":"2020-04-19T22:13:48","slug":"troubleshooting-landing-page-crontab-curl-tutorial","status":"publish","type":"post","link":"https:\/\/www.rjmprogramming.com.au\/ITblog\/troubleshooting-landing-page-crontab-curl-tutorial\/","title":{"rendered":"Troubleshooting Landing Page Crontab Curl Tutorial"},"content":{"rendered":"
\"Troubleshooting<\/a>

Troubleshooting Landing Page Crontab Curl Tutorial<\/p><\/div>\n

Back earlier in December (2019) the crontab\/curl work of Landing Page Mobile Phone Crontab Curl Tutorial<\/a> must have been disrupted.<\/p>\n

\nWhy think that?\n<\/p><\/blockquote>\n

The RJM Programming Landing Page<\/a> dropdown of blog tutorial links was missing something … anyone, anyone<\/a>? … yes, Sunil Gavaskar<\/a> … its blog posting links. And how are they created each day … anyone, anyone<\/a>? … yes, Laika<\/a>, look down to Landing Page Mobile Phone Crontab Curl Tutorial<\/a> to read all about it.<\/p>\n

\nDo you think it would be good to try, via ssh access what crontab\/curl automates on the RJM Programming web server, but on the command line?\n<\/p><\/blockquote>\n

Excellent suggestion!<\/p>\n


\n$ curl HTTP:\/\/www.rjmprogramming.com.au\/PHP\/tutorial_options.php
\n<b>Error 3: Error writing file '\/tmp\/MYpxajPE' (Errcode: 28)<\/b><option selected='selected' value=' '>Tutorials (show blog, toggle sort order) ...<\/option>
\n<\/code><\/p>\n

\nAh! Do you think a diskspace examination of the \/tmp disk on this Linux web server would reveal anything useful?\n<\/p><\/blockquote>\n

You’re on a roll!<\/p>\n


\n$ df -k \/tmp
\nFilesystem 1K-blocks Used Available Use% Mounted on
\n\/usr\/tmpDSK 544256 513316 3292 100% \/tmp
\n<\/code><\/p>\n

\nWow, that’s bad! Do you think a file listing examination of the \/tmp disk on this Linux web server would reveal anything useful?\n<\/p><\/blockquote>\n

Have you been reading the script to all this?<\/p>\n


\n$ ls -l \/tmp
\ntotal 490508
\ndrwxrwxrwt 114 owner group 294912 Dec 2 03:54 .\/
\ndr-xr-xr-x. 25 owner group 4096 Dec 1 04:49 ..\/
\n-rw-r--r-- 1 owner group 32 Aug 9 2014 adminer.key
\n-rw-r--r-- 1 owner group 27 Aug 26 2014 aexception.is
\ndrwx------ 2 owner group 4096 Jun 16 2018 ansible_Hinjci\/
\n-rw-r--r-- 1 owner group 225887 Dec 2 02:04 basegetmelist.is
\n-rw-r--r-- 1 owner group 225887 Dec 2 02:01 basegetmelist.wis
\n-rw-r--r-- 1 owner group 43615 Jan 18 2015 basegetmelist.wisold
\n-rw-r--r-- 1 owner group 17099 Apr 8 2014 BatsAndBall.tcl
\n-rw-r--r-- 1 owner group 130989 Apr 23 2018 CDB-simple_after_LibreOffice.htm
\n-rw-r--r-- 1 owner group 20624 Jun 4 2015 check-mk-agent.rpm
\ndrwx------ 2 owner group 4096 May 11 2019 clamav-030407a30e41469e50714e18d06cfef6\/
\ndrwx------ 2 owner group 4096 Nov 10 2018 clamav-05c7969f7c92127c0a67d1e1e030ff7a\/
\ndrwx------ 2 owner group 4096 Jun 16 2018 clamav-0c9c5d08dd88487d2db8ab0d89844a98\/
\n# more follows ...
\n-rw------- 1 owner group 20041728 Dec 1 02:32 magick-299501ChMis2EYamN
\n-rw------- 1 owner group 31457280 Dec 1 02:32 magick-29950E5FvDZxbZUTb
\n-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30009gmKHkoN2pnsz
\n-rw------- 1 owner group 31457280 Dec 1 02:32 magick-30011B1tagrDAr7pR
\n-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30011VUw-Xwaqo2lT
\n-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30043kgDfzEiS4YhO
\n-rw------- 1 owner group 31457280 Dec 1 02:34 magick-30089dJLmu7BUm4Cw
\n-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30133nUvTOGPaB5XG
\n-rw------- 1 owner group 31457280 Dec 1 02:33 magick-30133qaJFxqr0Kbgr
\n-rw------- 1 owner group 7234560 Dec 1 02:45 magick-33938vgJrHboBpWeI
\n-rw------- 1 owner group 31457280 Dec 1 02:59 magick-36847TC-kxmkRqR4V
\n-rw------- 1 owner group 31457280 Dec 1 03:00 magick-36847YUdME4k4jgwh
\n-rw------- 1 owner group 31457280 Dec 1 02:59 magick-368918jQfT0gouz5z
\n-rw------- 1 owner group 31457280 Dec 1 03:00 magick-36891NSRHciA7uHb8
\n-rw------- 1 owner group 31457280 Dec 1 03:00 magick-3707285W0V0oGoH97
\n-rw------- 1 owner group 31457280 Dec 1 02:59 magick-37072lAYUvoSex9wk
\n-rw------- 1 owner group 31457280 Dec 1 03:00 magick-37073XLevidVmQCPl
\n# more follows ...
\n-rw-rw-rw- 1 owner group 87155 Mar 26 2019 minify_0dca26f4ae4988d406871896dc562350
\n-rw-rw-rw- 1 owner group 15260 Mar 26 2019 minify_0dca26f4ae4988d406871896dc562350.gz
\n-rw-rw-rw- 1 owner group 85916 Mar 26 2019 minify_2972bf58405bbbc339884b94935eb6bf
\n-rw-rw-rw- 1 owner group 17938 Mar 26 2019 minify_2972bf58405bbbc339884b94935eb6bf.gz
\n-rw-rw-rw- 1 owner group 12447 Mar 26 2019 minify_67325ffb44e64f839c31db871aa8f6de
\n-rw-rw-rw- 1 owner group 2824 Mar 26 2019 minify_67325ffb44e64f839c31db871aa8f6de.gz
\n-rw-rw-rw- 1 owner group 185518 Mar 26 2019 minify_f8189fe934d7fecd140658eef38fcf74
\n-rw-rw-rw- 1 owner group 34816 Mar 26 2019 minify_f8189fe934d7fecd140658eef38fcf74.gz
\n-rw------- 1 nobody root 2227 Apr 25 2014 mNQtmxm4OS
\n-rw-r--r-- 1 owner group 0 Apr 28 2015 msans.ans
\n-rwxr-xr-x 1 owner group 741 Apr 28 2015 msans.ksh*
\n-rw------- 1 mysql mysql 0 Jun 19 2018 MYchYpJx
\n-rw-r--r-- 1 owner group 0 Sep 12 2017 mysocket1337.tmp
\n-rw-r--r-- 1 owner group 386 Oct 21 2014 no.worries
\ndrwxr-xr-x 2 owner group 4096 Oct 21 2014 okold\/
\n-rw------- 1 owner group 0 Jan 11 2017 pdfFafOZA
\n<\/code><\/p>\n

\nThose magick-* files seem problematic! Do you think a look around the net would reveal anything useful?\n<\/p><\/blockquote>\n

Astounding! Look at what are ‘\/tmp\/magick-*’ files? – ImageMagick<\/a> …<\/p>\n

\nWhen IM can’t do its work in main memory, it uses disk. If the process completes normally, the temporary disk files are deleted. If the process fails, these files are often not deleted.
\n
\nIf you have no IM processes running, you can safely delete these files.\n<\/p><\/blockquote>\n

\nKnowing that, and seeing the dates of those problematic \/tmp\/magick-* files, am I right in thinking you’ll tabulate a short term and long term strategy remedy to the problem?\n<\/p><\/blockquote>\n

And do fish swim?!<\/p>\n\n\n\n
Short term strategy<\/th>\nLong term strategy
Within ksh shell script that crontab runs each minute, add (allowing 40 minutes before saying the ImageMagick must have failed) …<\/th>\n<\/tr>\n
\n
\n$ rm -f \/tmp\/magick-*
\n$ df -k \/tmp # Sanity check gives \"\/usr\/tmpDSK 544256 24428 492180 5% \/tmp\" ... much better
\n$ curl HTTP:\/\/www.rjmprogramming.com.au\/PHP\/tutorial_options.php
\n<\/code>\n<\/td>\n		\n
\nksh -c 'for i in `find \/tmp -name \"magick-*\" -atime +40m`; do rm -f $i; done'
\n<\/code>\n<\/td>\n<\/tr>\n<\/table>\n

Feel free to see a PDF stream of consciousness<\/a> presentation of us dealing with this issue.<\/p>\n

\n

\n

Previous relevant Landing Page Mobile Phone Crontab Curl Tutorial<\/a> is shown below.<\/p>\n

\"Landing<\/a>

Landing Page Mobile Phone Crontab Curl Tutorial<\/p><\/div>\n

That work with the RJM Programming Landing Page we were last looking at a couple of days ago (with Landing Page Mobile Phone Tutorial<\/a>) was all fine and good, but the introduction of a new HTML indexmobile.html<\/i> for mobile phone usage, specifically, needs melding into existant web server arrangements.<\/p>\n

For the most part “web server arrangements”<\/i> has little to do with HTML and Javascript, which is usually client side work, unless, perhaps you involve Ajax<\/a>.<\/p>\n

But even ignoring Ajax, and ignoring server side languages like PHP (or ASP.Net) … remember …<\/p>\n

\n

As we continue in this series of tutorials we don’t pretend to be experts on aesthetics but we will show you some HTML and Javascript (no PHP, deliberately (read Linux sudo nohup Watchdog Primer Tutorial<\/a> to get this in context, for us)) functionality ideas that could supplement some great graphical design ideas you have to start something yourself, perhaps.<\/p>\n<\/blockquote>\n

… from Landing Page Primer Tutorial<\/a> … how can HTML (with its associated Javascript) make use of “web server”<\/i> anything much other than being a place it gets stored?<\/p>\n

Well, that web server is a server … a computer … with its own processes … ours is a Linux web server “computer”<\/i> … with batch processing capabilities … easiest to access via crontab<\/a>, and we first talked about how the RJM Programming Landing Page could be supplied with an HTML select element filled with RJM Programming blog tutorials with the most recent ones up the top, via the “crontab”<\/i> running of a PHP web application that reads the WordPress MySql database to derive the contents of this HTML select element plonked into the Landing Page when we presented More on Linux Crontab Curl Tutorial<\/a> as shown below.<\/p>\n

So here we have a before<\/i> and after<\/i> scenario … that doesn’t involve hair nor weight loss<\/font> … so here is the crontab before (which we’ll keep but add another similar process onto) combining its talents with PHP and curl<\/a> … the “CC dynamic duo”<\/font> …<\/p>\n


\n59 23 * * * curl http:\/\/www.rjmprogramming.com.au\/PHP\/tutorial_options.php 2> \/home\/rjmprogr\/public_html\/PHP\/tutorial_options.bad
\n<\/code><\/p>\n

… but we need to tell you a great thing about curl<\/i> … you can access, easily, a single $_GET[]<\/a> PHP parameter, because Linux doesn’t think “?” is very special (but it does have a “background process meaning” for “&” making it hard to use curl<\/i> with more than one $_GET[] PHP parameter involved) … leading us to our new bold<\/b> crontab<\/i> record below to cater for “indexmobile.html” updating …<\/p>\n


\n59 23 * * * curl http:\/\/www.rjmprogramming.com.au\/PHP\/tutorial_options.php 2> \/home\/rjmprogr\/public_html\/PHP\/tutorial_options.bad
\n57 23 * * * curl http:\/\/www.rjmprogramming.com.au\/PHP\/tutorial_options.php?mobile=mobile 2> \/home\/rjmprogr\/public_html\/PHP\/tutorial_options.bad<\/b>
\n<\/code><\/p>\n

… and what needed doing with the PHP?<\/p>\n

Here is the downloadable PHP programming source code you could call tutorial_options.php<\/a> changed for the purposes of being capable of updating “indexmobile.html” as per this link<\/a>.<\/p>\n

The effect on mobile phone usage of the RJM Programming Landing Page tutorial dropdown (HTML select element) list of RJM Programming blog tutorials (updated daily) is seen with today’s tutorial picture<\/a>.<\/p>\n

\n

Previous relevant More on Linux Crontab Curl Tutorial<\/a> is shown below.<\/p>\n

\"More<\/a>

More on Linux Crontab Curl Tutorial<\/p><\/div>\n

Sometimes for a website you only want to do a web server task once or a few times a day, and you, ideally, wouldn’t want to run into the remote possibility that two people could perform the task at precisely the same time. This scenario happens often when using server-side languages like ASP.Net and PHP. If it didn’t matter that two people might do the same thing at once then you could do the job within your web-based PHP, and you may consider using a date and time test in your web-based PHP to do your “once or a few times” task in your PHP. This latter idea has the advantage that file ownership issues will probably not crop up because the web user will be manipulating any media or HTML files you may be dealing with, rather than the administrator user that will probably be overseeing your “once or a few times” cron<\/a> job that we show you here today.<\/p>\n

So the task today is to keep the rjmprogramming.com.au’s landing page<\/a> “Tutorials” dropdown up to date with the latest blog postings, rather than having to manually attend to this with interactive (HTML code) edits and (s)ftp<\/a> uploads. You may be wondering why not just write the landing page in PHP, and this is a fair point, but the strategy here at this domain is that the landing page and its “fellow” webpages should be independent of MySql, which, under big loads, can go down, and here we use a watchdog<\/a> to fix that, should it happen. Previously we had a very similar task you can read about with the Linux Crontab Curl Primer Tutorial<\/a> below. It is so similar, let’s use curl<\/a>, PHP and crontab to do this task like with that previous task. So this is command line, and this is batch work, but that worry in other types of batch work where you need to worry about where you are as you do any file management, is handled quite well by the wrapping of the job in curl, which takes as its argument a URL, so, implicitly, the place to work with file management, is handled by curl and the PHP we write (specifically where the code uses dirname(__FILE__)<\/i>).<\/p>\n

Here is the downloadable PHP programming source code you could call tutorial_options.php<\/a> and you will see, if you examine the code that it is very specific to conditions here at this rjmprogramming.com.au domain, so please be aware of that, but maybe its concepts can help you out with something else you are tackling. A list of the specifics you would want to address (should you adapt it for your own purposes) include:<\/p>\n