# how would i make a search engine?



## Fitseries3 (Oct 31, 2009)

im a total noob here but i have an idea that would help make things alot simpler if i could get this to work.

how would i make a search engine to search a relatively narrow group of specified websites?


----------



## vaiopup (Oct 31, 2009)

May be a good question to ask at MJ-12 forums

http://www.majestic12.co.uk/forum/


----------



## Solaris17 (Oct 31, 2009)

well what are you looking for? like using a search engine on a web site? or like making a program and running asearch that will seach your specified sites?


----------



## Fitseries3 (Oct 31, 2009)

well i had the idea of a toolbar for firefox but i can host a site with my searchengine once i get it going.


----------



## Solaris17 (Oct 31, 2009)

Fitseries3 said:


> well i had the idea of a toolbar for firefox but i can host a site with my searchengine once i get it going.



ahhh ok ill do a little digging though making your won search engine can be quite difficult. however if you want to take a simpler way out google lets you make your own search engine and you can specify what sites and stuff to let it search. and if your crafty im sure you could remove the google code that they probably thriow in their. id take a look anyway...here is the link tell me if you can see it or not you should but im currently in my adsense account and stuffs.

http://www.google.com/cse/?v


----------



## Fitseries3 (Oct 31, 2009)

hmm... not working like i need it to. 

too general. 

i want to search multiple forums for threads. how would i do that?


----------



## Disparia (Oct 31, 2009)

At it's simplest, it could just be a script to pass on your search terms to multiple forum search pages through cURL and return the results as-is.

Getting more into it, could have a database for caching results as well as the utilization of advanced search features (especially date range) of each forum for faster returns. Depends on if it's something just for you, or if you want to offer a more polished product.


----------



## Fitseries3 (Oct 31, 2009)

i would like to get it to where anyone can use it. but its proving to be quite a task.


----------



## Corrosion (Oct 31, 2009)

Would be cool to have a search bar for tpu.


----------



## mlee49 (Oct 31, 2009)

+1 to that, the search function is a bit gawky and doesnt always find what I'm looking for.


----------



## temp02 (Oct 31, 2009)

The first step in making a complete search engine (like Google) is building a Web Crawler, and that isn't very hard to do, the hardest thing to do is building a really good one (remember Yahoo vs Google days?), for that you'll have to, at least, attend some Probability classes.

As for the TPU search bar (for Firefox, which is very easy to implement), as long as *W1zzard* removes that *anti bot* input box and replaces the "post method" on the search page by URL parameters it should be easy to do (he will probably not do this for security/spaming issues).


----------



## W1zzard (Oct 31, 2009)

you spider all the content on the websites you want to search and store it somewhere, then you search in that data when the user sends you a query and you return the results.

getting the basics to work seems pretty simple. the text content of all posts on tpu forums consumes only about 650 mb


----------

