[Interest] Small survey on necessary Qt Container size

[Roland Hughes]

On 9/6/22 05:00, A. P?nitz wrote:
> The question is really meant as conjunction, i.e. I'd like to count
> only setups meeting both criteria at the same time:
>
>     1./Some/  relevant data set is really > 1e9 entries,
>
>     2. It really needs to be a/Qt/  container because of some Qt container
>        feature (e.g. reference counting or e.g. because of Q(5)List's
>        "indirect" storage etc) that a std:: container does not offer
>        out-of-the-box and it/needs/  to be like that, i.e. there is no
>        simple / straightforward replacement like using std::*, and
>        benefits do not exceed drawbacks like more expensive write
>        access.

On 9/6/22 05:00, Thiago Macieira wrote:

> Or foresee needing them in the next 10 years.

For applications contained entirely on the 
x86-wanna-be-a-real-computer-one-day-when-I-grow-up platform, only once.

For applications that would run on above as front ends for systems 
hosted on real computers, I've lost count. Qt simply isn't even 
considered anymore. Given that Ford is laying off 8,000 and has dumped 
manufacturing cars, Qt might want to consider the real computer market.

https://www.foxbusiness.com/economy/ford-lay-off-8000-workers-shift-ev-production

They also might want to consider making all of the QSqlTable based stuff 
operate outside of the main event loop while only nibbling data from 
cursors. Dumping the failed methodology of MVVM in favor of recreating 
the proper cursor.

Of the database engines listed for 6.x, only PostgreSQL is actually used 
for data warehousing, at least by companies that are smart enough to not 
put their critical data in the cloud. DB2 and the rest are used for 
PRODUCTION databases. They get trimmed. Despite MariaDB touting itself 
as a Data Warehousing product, I've never seen anyone serious use it for 
that.

https://mariadb.com/kb/en/data-warehousing-techniques/

Oracle gets used in many and was the first to have a database where 
tables could exceed 2TB in size. It was built for Boeing. Informix has 
long been used for Data warehousing and IBM now has Netezza appliances.

The x86-wanna-be world is oohing and aahing over Mongo and NoSQL. The 
Fortune 1000 world is dusting off PICK BASIC and Mumps

https://www-50.ibm.com/partnerworld/gsd/solutiondetails.do?solution=49945&expand=true&lc=en

Look under the hood of CoudAnt some time.

At any rate,

A really big software development market right now is in creating front 
ends and reporting tools for locally hosted data warehouses. Because of 
the SQL classes needing to exist in the main event loop and because 
response time from a data warehouse having hot and cold storage could be 
minutes, Qt cannot be chosen. Good thing, because with Qt's size limits, 
it also cannot be chosen.

Real requirement:

Keller MBA graduate wants a part evaluation spreadsheet (they always 
want a spreadsheet) where they can choose a part number, and have the 
spreadsheet filled in with every order ** for the entire part 
supercession chain **

The 380AMP alternator they make today will fit any truck they made from 
1970 through to today. The part number will have changed hundreds of 
times (and is stored in a circular referencing table because you don't 
get rid of trapped inventory by deleting a supercession, you superceed 
back to the original part number)

In their spreadsheet they want to see every dealer order for a 380AMP 
alternator from the beginning of time. Then they want little check boxes 
and radio buttons so they can weed out (or select) based on warranty or 
"just an order" or, I forget which else.

Last I heard, the data warehouse table for parts order detail lines had 
over 30 billion rows. That was close to ten years ago.

Twenty years ago Boeing had to solve the same problem for airplane parts.

As of right now, Informix can store a maximum of 4,278,189,825 rows in 
each table fragment and a maximum of 2047 fragments in a single table. 
ROWID is only unique within a fragment.

https://www.ibm.com/support/pages/what-maximum-number-pages-table-or-fragment-what-maximum-number-rows-page

The XPS version of Informix allows 32766 fragments and ROWID is only 
unique within a fragment.

How many of you currently have Qt applications that rely on ROWID being 
unique???


If Qt wants to find a market, now that the automotive market has 
disappeared and phones have turned their back on Qt it has to become the 
front end that can communicate with what the real computers can throw at it.

**You can't fit everything into RAM and you have to do I/O outside of 
the main event loop **


Once you achieve that you should develop classes for the M database, 
both OpenSource and the one from InterSystems.

https://en.wikipedia.org/wiki/MUMPS

https://www.intersystems.com/

The largest medical records systems in America, and possibly the world, 
use the Mumps database. Right now they have one choice for GUI 
development tools. Medical records systems are not going away.

Then you can tackle a class for the IBM Universe database. There is 
enough revenue there for IBM to keep an entire division alive around the 
product. Most (all?) of you have never heard of PICK BASIC or IBM 
Universe but I worked with PICK BASIC when Advanced Revelation first 
came out. That's a very significant niche that currently has only one 
vendor for tools to develop applications with. They cannot replace what 
they have for a database with anything else.


Sorry for the lengthy answer, but the single-thread-i-ness of Qt and 
tiny x86 size design mentality means it can never be used for business 
desktop applications which increasingly must hit data warehouse stores. 
Gone are the days where a company sets up one PostgreSQL database and 
feeds all of their orders into it for eternity. Today they keep a light 
and nimble less than 5 year history in the PRODUCTION database and 
everything else is in the warehouse.

You are seeing the same thing with your stock brokers. Fidelity and many 
others only keep 2-3 years of transactions on-line. If you need older 
transactions for an audit or whatever, you have to submit a report 
request to be run against the data warehouse.

-- 
Roland Hughes, President
Logikal Solutions
(630)-205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog