Wednesday, October 14, 2009

Control firefox from PHP?

Internet Explorer can be controlled from a COM object interface, and therefore from PHP (i.e. any scripting language that has COM support).

But is there a way to get script control of firefox? Ideally I'm looking for a platform-independent solution, and something I can use from PHP. Google is not helping (PHP's dominance as a server-side technology overwhelms the client-side related hits).

Here is my dream PHP script:

$firefox=new FirefoxInstance();
$firefox->set_url("http://dcook.org/work/");
$firefox->wait_until_fully_loaded();
$links=$firefox->get_links();
$found=false;
foreach($links as $id=>$info){
if($info['text']=="MLSN"){$firefox->click_link($id);$found=true;break;}
}
if(!$found)echo "MLSN link is missing...\n";

Or:

$form=$firefox->get_form("login");
$form->set("username","guest");
$form->set("password","guest");
$form->submit();

Et cetera. I.e. I'm talking about operating firefox the same way a user does; I know I can grab the raw HTML, parse it, etc. all from PHP, but that doesn't test a web site the same way clicking links in a browser does. Especially web pages with javascript, iframes, AJAX, etc.

(I'd heard of XPCOM but, if I've understood it correctly, it is a library to build firefox and its extensions, not something to control firefox? It also has no PHP bindings.)

BTW, going back to controlling IE from the COM interface, I don't suppose anyone has seen a detailed tutorial on how to use it to fill out and submit forms? I only ever see simple examples of how to set the URL, but I believe full control should be possible.
Dec 16th 2009 UPDATE: This came to the top of my to-do list so I read the MSDN docs on the Internet Explorer COM object, and now it is my understanding that I cannot manipulate and submit forms via the COM object. None of the example usage even hinted at doing this.

14 comments:

keith.s.wilkinson said...

Found the following for IE and Firefox

keith.s.wilkinson said...

There's also MozDev

darren said...

Hi Keith,
Thanks. The two firefox links were top pages of the firefox forum and developer's web site respectively.
Do you still have the actual URLs of the pages you found? Or remember the keywords you used to find something?

keith.s.wilkinson said...

My thinking was that the best way to have your ideas seen by other people who either want to do the same thing or who have already done something similar would be to post in the appropriate forum ;-)

keith.s.wilkinson said...

Firefox allows you to save a large number of open sites (tabs) as a default startup set. But I'd like to be able to choose between multiple startup sets (different sets of web sites for different purposes) i.e. I'd like to be able to create not just individual bookmarks but also bookmark tab sets that can be opened with one click.
One thing that may make it difficult to operate Firefox entirely under program control is that Firefox seems to prevent certain sites from loading automatically under certain circumstances, in the interest of preventing crashes. It seems that Google probes all the open tabs to see if Google services are open on multiple tabs -- so if you log in on one tab then you'll automatically be logged in on other tabs; this probing is a very heavy load on Firefox when it is starting up, and causes crashes; it seems to me that Firefox often prevents Google from loading and logging me on in this case (I have to reload the page manually).

keith.s.wilkinson said...

Why not make a Mozilla Concept proposal: "I'd like to be able to control Firefox from PHP in order to (explain purpose). Ideally I'd like to be able to..."

Алексей said...

Darren, I found a way to access Firefox from PHP. First of all, we must setup extension MozRepl to access Firefox from telnet connection. It allows to evaluate any javascript code in telnet console. You can connect from Putty or any other telnet client, and, of course, from PHP, because it allows to create a telnet connection. Then we have to write a wrapper library (I did not found ready solutions) and voila! - any javascript written into a socket is executed by Firefox. You even can read data from there, wrapping it into JSON or any other way. Good luck!

darren said...

Thanks Алексей!!
I've just installed MozRepl and have been testing it from telnet. After a bit of trial and error I can load a page, fill in data and submit the form, as well as call any of the javascript functions on that page.

Here is how to discover the forms in a page:
repl.enter(content)
document.forms

[object HTMLCollection] — {0: {…}, 1: {…}, 2: {…}, length: 3, item: function() {…}, namedItem: function() {…}}

There are three forms on this page, so that is correct. To submit the main form I can then do:

document.forms[2].mytestfield.value="test"
document.forms[2].submit()

One more note: it operates on whichever firefox window/tab you have active when you make the telnet connection.

Алексей said...

No, you can control a whole browser, not only active tab. You can open new tabs, control any of already opened, and even control main chrome window, get access to UI, loaded extensions and any object that Firefox contains, like regular extension can do. Look at global Firefox object gBrowser that gives you a lot of methods to access several tabs. For example, to create a new tab with google.com inside you can do:

gBrowser.selectedTab = gBrowser.addTab("http://google.com")

And here you can find a lot of code snippets that can be very useful for several tasks.

darren said...

I've now done a full project with MozRepl (combined with HttpFox), and will blog about it soon. Basically it works, is more powerful, and is so incredibly more stable than the IE+HttpAnalyzer combination I was trying to use before.

Thanks for the links Алексей, that is exactly the kind of docs I was missing (you can use repl.look() to work out lots of stuff, but that gets a bit tedious).

darren said...

Some more usage examples:
http://firefox-extension.wikidot.com/examples-of-code

And this is how to clear the cache (from a firefox extension; I'm assuming mozrepl will let me do the same):
http://github.com/choltz/clear-cache-button/blob/v.9a/chrome/content/clearcache/overlay.js

Keith said...

Have just noticed your article "Firefox For Control Freaks" on this, in the March issue of PHP Architect ;-)

darren said...

Hello Keith. Yes, that php|a article started off as a blog post. But it got a bit bigger and in-depth so I offered it to php|a. I have permission to post it as a web article, so I must get around to that soon.

BTW, this is a good place to mention that while researching that article I discovered Selenium; for most (though not all) cases where you need to control Firefox it is a better choice than MozRepl. It required me to throw half my article away and rewrite it!

Keith said...

Firefox Developers Conference (Japan) 2010
http://mozilla.jp/events/2010/fxdevcon/