Twilight SEO

Standing at the border line. Beautiful color spectrum of Search Engine Optimization. White, Gray, Black Hat SEO learning by doing.

PHP4: How to Steal from Yahoo! (another Web Screen Scraper)

Posted in php4 by ruru on the February 11th, 2008

If you've been wondering on how to scrape some information from your favorite website. This web screen scraping techniques has already there since dinosaur age. Here's an example on how to do it in PHP 4. Thanks to:

  1. Mozilla Firefox and Firebug for spying and taking overall bird view so we could design a strategy.
  2. The good old snoopy-php project which will do his works as http client, snipping and sucking the whole page.
  3. And also phphtmlparser that make dirty HTML parsing work a lot more easy, taking down the enemy element by element.

First, our target operation is to grab a list of currently hot box office movies from Yahoo! Movies.
Target located: http://movies.yahoo.com/mv/boxoffice/ ..... locked on!!!

We need to see the HTML layout of the page, using Firefox and Firebug
getting html layout for screen scraping yahoo movies

Place your sight on the right side of screenshot.
The area we want to steal started with Top Movies and ended with Top Cast/Crew'.
Here's part of the code you'll see later.

PLAIN TEXT
PHP:
  1. ...
  2.             incIf($step1, 0, $parser->iNodeValue == 'Top Movies');
  3. ...
  4.             incIf($step1, 1, $parser->iNodeValue == 'Top Cast/Crew');
  5. ...

Now look at the bottom part. The full element tree starting from the root html tag, and ended in b tag.
xpath for screen firefox and firebug scraping yahoo movies
Decide that b < font < a < td should enough to distinct and separate the element from others.
So we will use a variable $step2 for digging. If $step2 == 0 and current element is td, we set $step2 into 1. If $step2 == 1 and current element is b, increment it. This $step2 continues to dig further to font and until we reach the treasure box in b tag. Finally, print out what inside treasure box and go up again to the surface, set $step2 to 0.

Now it's time for the full code guys.
If you're too lazy to copy paste it, just download the source code of web scraping tutorial (PHP 4).

PLAIN TEXT
PHP:
  1. <?
  2.     // Let's hire both experts
  3.     include ('Snoopy.class.php');
  4.     include ('htmlparser.inc');
  5.  
  6.     // Move and dig deeper
  7.     function incIf(&$step_counter, $current_step, $condition) {
  8.         if (($step_counter == $current_step) && ($condition))
  9.             $step_counter = $current_step+1;
  10.     }
  11.  
  12.     // If it's deep enough, take it and leave;
  13.     function doIf(&$step_counter, $current_step, $condition) {
  14.         if (($step_counter == $current_step) && ($condition)) {
  15.             $step_counter = 0;
  16.             return true;
  17.         }
  18.         return false;
  19.     }
  20.  
  21.     // C'mon snoopy suck that page
  22.     $snooper = new Snoopy();
  23.     if ($snooper->fetch('http://movies.yahoo.com/mv/boxoffice/')) {
  24.         // Pass the page to HtmlParser, and let him do his work
  25.         $parser = new HtmlParser ($snooper->results);
  26.         $step1 = 0; $step2 = 0;
  27.         echo "TODAY's BOX OFFICE\r\n<br/>";
  28.         while ($parser->parse()) {
  29.             incIf($step1, 0, $parser->iNodeValue == 'Top Movies');
  30.             incIf($step1, 1, $parser->iNodeValue == 'Top Cast/Crew');
  31.             if ($step1 == 1) {
  32.                 incIf($step2, 0, $parser->iNodeName == 'TD');
  33.                 incIf($step2, 1, $parser->iNodeName == 'A');
  34.                 incIf($step2, 2, $parser->iNodeName == 'FONT');
  35.                 incIf($step2, 3, $parser->iNodeName == 'B');
  36.                 if (doIf($step2, 4, $parser->iNodeType == NODE_TYPE_TEXT))
  37.                     echo $parser->iNodeValue."\r\n<br/>";
  38.             }
  39.         }
  40.     }
  41. ?>

Tags: php4, screen scraping, code example

0 Comments

Cake PHP: Simple Contact Form Tutorial!

Posted in cakephp by ruru on the February 8th, 2008

This guide provide a quick tutorial covering simple contact form for CakePHP v1.2, based on this post
Download code example here! CakePHP 1.2 Simple Contact Form.zip

1. Create the VIEW in cake/app/views/contact/contact.ctp

PLAIN TEXT
PHP:
  1. <?php if (isset($success)) { ?>
  2.     <p><?= $success?></p>
  3. <?php } else { ?>
  4.     <?php if (isset($error)) { ?>
  5.         <p><?= $error?></p>
  6.     <?}?>
  7.     <?php echo $form->create(null,array('action' => 'contact', 'enctype' => 'multipart/form-data')); ?>
  8.     <p>
  9.         <?php echo $form->label('Page.name','Name: ',null); ?>
  10.         <?php echo $form->text('Page.name', array('size' => '25'))?>
  11.     </p>
  12.     <p>
  13.         <?php echo $form->label('Page.email','Email: ',null); ?>
  14.         <?php echo $form->text('Page.email', array('size' => '25'))?>
  15.     </p>
  16.     <p>
  17.         <?php echo $form->label('Page.message','Message: ',null); ?>
  18.         <?php echo $form->textarea('Page.message', array('cols' => '40', 'rows' => '10'))?>
  19.     </p>
  20.     <p>
  21.         <?php echo $form->submit('Send email')?>
  22.     </p>
  23.     <p>
  24.         All fields are required!
  25.     </p>
  26.     <?php echo $form->end();?>
  27. <?php }?>

2. Create Email COMPONENT, cake/app/controllers/components/email.php

PLAIN TEXT
PHP:
  1. <?php
  2. class EmailComponent extends Object {
  3.     function email($email, $name, $message) {
  4.         $f_mail = "name@domain.com";
  5.         $f_name = "WebMaster domain.com";
  6.         $f_message = "\nName : $name"; // start building the message
  7.         $f_message .= "\n\nE-mail : $email";
  8.         $f_message .= "\n\nMessage \n : $message";
  9.         $headers = "From: www.domain.com <administrator@domain.com>\n";
  10.         $headers .= "X-Sender: <admin @domain.com>\n";
  11.         $headers .= "X-Mailer: PHP\n";
  12.         $headers .= "Reply-To:$f_mail\n\n";
  13.         $subject = "Message from your website domain.com";
  14.         mail ( "$f_mail", "$subject", "$f_message", $headers ); // send the email
  15.     }
  16.    
  17.     // this function checks to see if the email supplied by the visitor is a valid one
  18.     function validMail($email) {
  19.         $pattern = "^[-_.[:alnum:]]+@((([[:alnum:]]|[[:alnum:]][[:alnum:]-]*[[:alnum:]])\.)+(ad|ae|aero|af|ag|ai|al|am|an|ao|aq|ar|arpa|as|at|au|aw|az|ba|bb|bd|be|bf|bg|bh|bi|biz|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|com|coop|cr|cs|cu|cv|cx|cy|cz|de|dj|dk|dm|do|dz|ec|edu|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gh|gi|gl|gm|gn|gov|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|in|info|int|io|iq|ir|is|it|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|mg|mh|mil|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|museum|mv|mw|mx|my|mz|na|name|nc|ne|net|nf|ng|ni|nl|no|np|nr|nt|nu|nz|om|org|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|pro|ps|pt|pw|py|qa|re|ro|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|sk|sl|sm|sn|so|sr|st|su|sv|sy|sz|tc|td|tf|tg|th|tj|tk|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|um|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)|(([0-9][0-9]?|[0-1][0-9][0-9]|[2][0-4][0-9]|[2][5][0-5])\.){3}([0-9][0-9]?|[0-1][0-9][0-9]|[2][0-4][0-9]|[2][5][0-5]))$";
  20.         // the regex for an email addres
  21.         if (eregi ( $pattern, $email ))
  22.             $ll = 1; else
  23.             $ll = 0;
  24.         return $ll;
  25.     }
  26. }
  27. ?>

3. Create the CONTROLLER, cake/app/controllers/contact_controller.php

PLAIN TEXT
PHP:
  1. <?php
  2. class ContactController extends AppController {
  3.     var $uses = array ( );
  4.     var $components = array ('Email' );
  5.     var $name = 'Contact';
  6.    
  7.     function contact() {
  8.         $this->pageTitle = 'Contact';
  9.         if (! empty ( $this->data )) {
  10.             $email = $this->data ['Page'] ['email'];
  11.             $name = $this->data ['Page'] ['name'];
  12.             $message = $this->data ['Page'] ['message'];
  13.             $error = 0;
  14.             // start filter vars
  15.             if (strlen ( trim ( $email ) ) <3) {
  16.                 $error = 1;
  17.             }
  18.             if (strlen ( trim ( $message ) ) <4) {
  19.                 $error = 1;
  20.             }
  21.             // verify email address
  22.             if ((strlen ( trim ( $email ) ) <3) || ($this->Email->validMail ( $email ) == 0)) {
  23.                 $error = 1;
  24.             }
  25.             if ($error == 0) {
  26.                 $this->Email->email ( $email, $name, $message ); //here is the email sent
  27.                 $this->set ( 'success', 'The message was sent <br />Thank you!' );
  28.                 // setting "succes" variable for displaying if email was sent.
  29.             } else {
  30.                 $this->set ( 'error', 'Please complete all fields' );
  31.                 // this is in case of error
  32.             }
  33.         } else {
  34.         }
  35.     }
  36. }
  37. ?>

FINALLY
After all that just test it in your favorite web browser http://localhost/cake/contact/contact

Tags: php4, cakephp, code example

0 Comments

CakePHP: How To Make Controller Without a Model?

Posted in cakephp by ruru on the February 8th, 2008
PLAIN TEXT
PHP:
  1. class MyController extends AppController
  2. {
  3.     // var $uses = null; works too, BUT MIGHT INTRODUCE BUGS
  4.     var $uses = array();
  5.  
  6.     function index()
  7.     {
  8.     }
  9. }

From: http://cakebaker.42dh.com/

Tags: php4, cakephp, code example

0 Comments

CakePHP: How to Add Theme Support?

Posted in cakephp by ruru on the February 7th, 2008

Now with CakePHP v1.2 (beta) we can easily add theme with its ThemeView class.
This guide about how to add theme into fresh CakePHP installation will be explained in very basic step-by-step . I learn it frrom here.
Updated: sample code provided

Download code example, extract inside CakePHP root.

First!
Create a file called app_controller.php inside cake\app\controllers.
Create CakePHP AppController
Edit the file and just with two lines of code, add theme support on your application:

PLAIN TEXT
PHP:
  1. class AppController extends Controller {
  2.     var $view = 'Theme';
  3.     var $theme = 'default';
  4. }

Second!
You need to put all your theme collections inside a folder called themed instead views. This folder doesn't exists yet, so go to cake\app\views and create new folder themed inside it.
Create themed folder inside view

Third!
Create another folder called default inside cake\app\views\themed, and place all your template files there.
This theme name is taken from

PLAIN TEXT
PHP:
  1. var $theme = 'default';

We can just create another folder and activate it very easily thanks to ThemeView class.

That's it now you have an application that support theme. Easy?! Yea, that was sooo Cake :)

Tags: php4, cakephp, code example

5 Comments

CakePHP: How To Make Model Without a Table?

Posted in cakephp by ruru on the February 6th, 2008
PLAIN TEXT
PHP:
  1. class ModelWithoutTable extends AppModel
  2. {
  3. var $useTable = false;
  4. }

From: http://cakebaker.42dh.com/

Tags: php4, cakephp, code example

0 Comments
« Previous Page

Cannot find your answer here?
Feel free to get in touch and ask PHP-ist anything, just anything :)