belgeindir.com (Under Development)

belgeindir.com was a web-based document search-engine project I worked on in 2013 and left in development. The core idea was to scan file-hosting sources, collect information about discovered documents in a database and let users search the resulting metadata.

This entry documents a development project rather than presenting the system as a completed production service.

Crawling and Metadata

Finding a document URL is only the first stage of a document-search system. File name, type, source address and any available metadata need a common representation. The same document or URL may also be discovered through more than one crawl path, making identity and duplicate handling natural data-model concerns.

Temporary remote failures, changed links and incomplete metadata are normal conditions for a crawler rather than exceptional assumptions.

Database and Search

Persisting discovered records separates data acquisition from interactive search. A user query can run against previously collected metadata instead of rescanning remote sources every time.

That separation is an early form of a pattern I later encountered in much larger systems: move expensive acquisition away from the online query path and serve requests from a prepared representation.

Related Desktop Project

My Köker Belgeindir 1.3 desktop application addressed the same document-retrieval problem from the user side, with search, preview, segmented downloads and a local library.

belgeindir.com explored the server-side direction: crawling, persistence and searchable metadata.

Retrospective

A current design of the same class would make robots policies, access conditions, rate limiting, canonical URLs, content-type validation, duplicate detection and index freshness explicit architectural concerns.

The surviving project record establishes the separation of document discovery, metadata persistence and search; features not completed in that work are not part of the project description.