FR version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
74% Positive
Analyzed from 4256 words in the discussion.
Trending Topics
#postgres#mysql#more#need#https#sqlite#postgresql#data#com#where

Discussion (121 Comments)Read Original on HackerNews
https://medium.com/revolut/recording-more-events-but-where-w...
It's easy to use Postgres poorly in ways that result in painful centralized bottlenecks.
(Obviously this is largely true for anything, but I think that in 2026, where there's also a lot of more-specialized/less-fleible but much-easier-to-scale well-supported mature alternatives, you should be VERY wary of making everything have a single central SPOF. What are your users going to expect in terms of maintenance windows, etc.)
I'd be cautious with articles that say things like "All cloud providers allow you to run (and scale!) PostgreSQL by clicking a single button." with no mention of how long that will take and what options should be set to make it faster, or the costs of those things.
then Sqlite works as well too. can run the whole thing on Cloudflare.
running Postgres isn't difficult. but dealing with a VPS for low traffic is a headache that's not necessary.
Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.
Size matters!
For most of the application out there elastic (or kafka or any other specialized tool) is just too much(and too costly). They can do fine with postgres or mysql. Actually, I'd argue that in a lot of cases even postgres is too much, probably sqlite is enough.
I tried to use rabbitmq for a small app, installed it, configured it and then it didn't work. Spent a day jumping through hoops getting it right.
Dumped it and used postgres, in half an hour. Worked like a charm.
PG is great and I work with it daily, but it's also not a problem to think about scale early and at least have a notional plan for what to and how to know when scale is becoming an issue in your system as you're designing it. Even PG is overkill and sqlite is more than enough for some of my projects.
There are a lot of specialized tools available, but you definitely don't need to put every one in your toolbox. Experience and observation help you make those edits -- and of course there's almost always room for improvement, but "good enough" definitely exists (until it doesn't anymore :D).
Then a system gets most of the benefits of not accidentally making it torturous to rearchitect for scale, without paying the headcount / complexity cost until it's needed.
If you’re just starting out, keep things simple. Otherwise, you probably already know exactly why you need something more than Postgres.
Nobody is saying that a huge ecommerce store with complicated filtered search logic should throw away their Elasticsearch cluster and switch to Postgres.
Or maybe you are looking at it from "just swipe your credit card at AWS" perspective, in which case "just use Postgres" articles are for a different audience.
Posts like this can be tiresome, yet the general consensus among developers seems to be "yeah, Postgre/SQLite is ok for 99% of the cases, but MY case is going to be in the 1%, because I am going to be the next Facebook".
I'm a huge PG fan, so I start everything with it, but SQLite is sane, and it generally has a happy upgrade path to PG If you need it.
Both are amazing technologies.
Anyway, it is a basic practice of keeping test and dev environment as close as feasible to production, to avoid missing issues and wrong assumptions.
https://sqlite.org/stricttables.html
I've done that before and the code was a mess. It works at the beginning but APIs do much more than piping data from the database. When you start dealing with ACL, external calls, code reuse, etc. It's just nice to have all the tools available to you from something like Python or Go.
* As a message queue: Only if your required features are very basic, like if you need cluster communication and run your own coordination protocol on top.
* High Volume Time Series: TimeScale works, but composes badly with other workloads on the same DB server ( from an operational perspective at scale )
* Vector Database: The same issues as with TimeScale.. PgVector for example lives in its own seperate "world" and the query planner sees it as a very opaque thing. Forget about adding vector storage to an existing high volume db, that must server other complex queries.. PGVector will either trash your caches, or take over your cpu so that workloads that used to work fine stall. This is IMO not a pgvector problem itself ( Kudos to those guys ) but rather that postgresql extension apis are not very good at exposing custom costs and tradeoffs to the system as a whole.
* Raw Data: Works for small files... why anyone would want to store large amounts of data in it would be a mystery, where it shines is accessing LOTS of small files where internal caching etc help a lot compared to raw filesystem access ( also a bit dependent on the filesystem and its tuning though )
* Microservice: If your service is ONLY exposing json data from some database model, then it should not exist at all IMO. Create a view and be done with it.
http://www.postgis.net
The relational model and sql force us to simplify our data models too much by eliminating relationships or just not dealing with them.
Think about a nested json blob from some web service api and storing it in SQL in normalized tables. No one is going to do that. Everything just becomes a denormalized mess and everything is hacked around it.
Instead of modeling things in the proper way, most of the world's data is modeled in a way so that we don't have join explosions in sql queries because they look scary. Data pipelines become these scary batch transformations where data is dumped somewhere else without anyway to trace back where it came from.
I encounter so many end-user applications and systems where you wonder: "why couldn't they allow a list of items here instead of a single box" or "why can't this reference this other thing".
Postgres has built in data types and functions that allows it to work with unstructured json documents, like you would use in MongoDB.
I don't think this is accurate and smells like an LLM hallucination to me.
From the Timescale/Tiger Data _pgvectorscale_ project's README:
> pgvectorscale builds on pgvector with higher performance embedding search and cost-efficient storage for AI applications.
I think this is where the confusion originates. I believe pgvector is primarily Andrew Kane (@ankane) and a cadre of OSS contributors.
As an aside, I've used Timescale/Tiger Data products and was very happy with them and their support. Their team was very engaged and responsive to all of our questions. They also fixed a pretty gnarly indexing bug I uncovered in pgvectorscale in an impressively short amount of time.
NVMe drives + Litestream + object storage(S3/R2..). sqlite simplifies things for the entire long tail of apps/services that aren't the Ubers and AirBNBs of the world.
Also, apparently Litestream could use a filesystem instead of an object store?
MySQL was generally faster, and while MyISAM was a bit limited Innodb was pretty powerful, and you had the choice. It was also simpler (imo) and avoided a lot of the xid/vacuum issues.
That said, still love Postgres. But at the time it started eclipsing MySQL, MySQL felt better positioned.
- Query planner is much worse (just yesterday I had to USE INDEX to sped up a query by 300x, I'm near-certain postgres would just have gotten it right) - Indexes are much more limited: no GIST, no GIN - No transactional lock (`pg_advisory_xact_lock` in postgres). This one was very surprising, it's a really useful thing and I had to implement it myself as a lock table
PostgreSQL also had more features back then, e.g. the JSON support is very nice if you need to do anything that doesn't neatly fit into the relational model.
Also postgres is a "proper" db, so I'm glad it generally won out.
MySQL caught up as well as far as I know, but it still may have some poor defaults that are widely used.
During the dot com era it was common to develop and launch on MySQL with the intention of migrating to something else if they became successful (though your typical LAMP stack developer regarded Oracle and SQL Server as being deeply 'weird', so many were willing to stick with MySQL despite the well-known limitations of MyISAM).
From where I'm standing, it seems that PostgreSQL became clearly preferable for new projects from the mid 2000s onwards, but it was only the Oracle acquisition that began to push existing users off MySQL.
Another thing: those were times when web applications were practically 99% reads, and not so great ACID was a non-issue.
Postgres is OK, but it has really a lot of quirks that are not that obvious.
I use a SQL databases as needed. I've used Postgres, Sqlite, Duckdb, Json files with AWS Athena, Oracle enterprise for ERP systems (a multitude of schemas and objects with interoperability), and others.
I'm currently, deploying Duckdb with AWS S3 Tables (Iceberg) to see how it fits for a use case I have.
IT is great and always changing. Keep trying new things.
Cheers
The job queue runs on the cache db, scheduling jobs to move data from bigquery into postgres. It’s pretty neat.
Now we’ve run into near-real-time requirements so clickhouse is getting thrown into the mix.
It’s pretty funny the lengths we go to to implement user facing analytics that’s basically just “you are visitor number X” from 1995.
- https://www.reddit.com/r/PostgreSQL/comments/1vbo5j8/raw_xml...
- your post did not have a single word on XML hence my comment
Different use cases have different scalability limits in PG, when you get to them you need to deal with them.
It would be perfect if it had somewhat transparent sharding, I mean a way to add another instance and distribute load without having to stop everything.
There are solutions, but they tend to be involved and when you get to that point in many cases it makes sense to just move that workload to something else that scales better.
For instance, for many simple needs MySQL is simpler than Postgres, with similar performance and consistency.
* No need for a connection pool, while many use cases with Postgres require PgBouncer and Co.
* Easy sort (and basic search) of multilingual text, because MySQL has case insensitive UTF8 collations.
* No need to VACUUM, which can be a hard problem (it was, the last time I used Postgres).
For full text search, I once worked on a project that considered several alternatives for this, including Postgres. Manticore Search was finally chosen because it was more performant, with better search results.
Not sure if I'm missing anything here, but if I want case-insensitive search I simply create an index on lower(column) and use that to query.
VACUUM is something you need to pay attention to at scale. And at that point you need to know your DB anyway and tune it. For smaller applications (and I don't mean only toy applications) it usually isn't an issue.
Or even pg_trgm trigram indexes, which are case-insensitive by default and support similarity search to accept typos and misspellings.
But I sure wish it was 'core' and we didn't have to worry about it potentially going away, becoming de-supported..
Not 100% sure about using PG for file system at scale however. I'd love to hear more on the challenges (vacuum, toast, anything else?)
https://github.com/seanwevans/pg_gpt2
https://github.com/seanwevans/pg_shell
https://github.com/seanwevans/pg_os
It is already a quite smooth experience, but there is work to make it even easier than that.
I work on Lakebase, opinions my own.
In most of the applications we build or maintain we use PostgreSQL + cloud storage. That's it. And it works very well, also for: storing JSON, full text search, as a queue, as a vector database. Other software may be better at providing those features, but I'm extremely happy we only need to understand & manage PostgreSQL.
Coming from storing billions of rows in Clickhouse and performing dozens of materialized operations I shudder to think about what that would look like in a DB that doesn’t even support declarative IVM.
No deamon. Single file per DB. Less configuration overhead.
Supporting more than 1 writer per process. Strict typing. Access controls. Replication at scale is more effecient than copy-pasting files (seems SQLite has improved on this one).
But for big, multi-writer work PG is the way to go.
I demonstrated this with ClickHouse: https://github.com/ClickHouse/pg_clickhouse/blob/main/doc/of...
We're working on a chdb based mechanism to copy to/from s3, maybe with fdw on top we can back table in s3
You can try similar things with pg_duckdb & pg_lake
I have decided to use clickhouse with that config because of missing S3 for logs and metrics for long term store.
multiple processes connected to it.
0: https://www.postgresql.org/docs/19/ddl-property-graphs.html
1: https://www.postgresql.org/docs/19/queries-graph.html
This is a not great start. I assume it refers to MyISAM which has not been relevant for over a decade at this point. InnoDB made different design than PG decisions and was (and perhaps still is) faster at point lookups.