[Development] Calendar Systems proposal

[Edward Welbourne]

Well, I've missed a long and lively discussion while I was away.  I've
now caught up with e-mail, digested the thread and done some (but by no
means enough) background reading; that's left me with some jumbled
notes.  This shall be long - let's start with the thread (and fit some
of those notes into answering it):

Soroush Rabiei:
>>>>>>>>>>> Please tell me more about your idea when you're back.

Take KCalendarSystem [0], replace its uses of QDate with a Julian day
number (as int [1]), tidy up a bit and have relevant Q(Date|Time)+
methods take one of these as an optional parameter (default: Gregorian)
that configures what they mean by things like year, month, week and day
within any of these.

[0] https://api.kde.org/4.x-api/kdelibs-apidocs/kdecore/html/classKCalendarSystem.html

[1] I notice that QDate uses a qint64 for Julian day, which is overkill;
a 32-bit integral type lets us have 5,879,610 years each side of our
epoch, which is ample, IMO.  KCalendarSystem seems to agree.  QDate
can't change that for its data member, nor do I propose to change its
method signatures, but I see no reason for the calendar system to use
anything bigger than int for Julian dates.

Of course, BC/SC reasons mean "optional parameter" won't (until maybe Qt
6) be an added parameter (with a default) at the ends of existing
methods but a (probably first and non-optional) parameter of overloads
of the existing methods.  Or something like that.

>>>>>>>>>>> I suppose adding new calendars by users, requires
>>>>>>>>>>> subclassing QDate?

A user calendar would be implemented by sub-classing the calendar system
virtual base class.  Instances of that would be passed to methods of
(stock) QDate to over-ride its default use of Gregorian.

>>>>>>>>>>> Or maybe somehow extend the enum that contains calendar
>>>>>>>>>>> systems[?].

An enum can only support the calendar systems for which Qt contains
code.  There shall always be calendars we don't support: it would be
best to ensure users can get support for that, one way or another.  An
app-developer with enough users who want a particular calendar system
should be able to add support for it without waiting for us to care
about that calendar system.

>>>>>>>>>>> I think adding the information on which calendar system is
>>>>>>>>>>> current date is on, can be added as a member (handlre/enum)
>>>>>>>>>>> to QDate.

{Backwards,Source} Compatibility (BC/SC, above) issues preclude that.

Sune Vuorela:
>>>>>>>>>> I think you need to touch quite some of the 'inner bits' of
>>>>>>>>>> date / time

That's fine - messing with internals is OK, it's public A[BP]Is that
can't be changed - except by adding methods.  Most of what I envisage us
changing for this would be turning some existing static functions in the
code into methods of a QGregorianCalendarSystem class, that implements
a generic QCalendarSystem API.

>>>>>>>>>> my two missing pet features:
>>>>>>>>>>  - Partial dates

I'm quite sure these don't belong in Q(Date|Time)+, for all that I can
see their value in calendar (and similar) apps.  There might be a case
for a recurrent event class, that supports all the funky things for
repeating calendar entries - e.g. the first Sunday following a full Moon
which falls on or after the Spring equinox, optionally with "full moon"
and/or "equinox" defined by some arbitrary set of rules rather than by
actual astronomical reality [2] - which would surely include partial
dates.  I encourage you to think about designing that and would be happy
to review any contribution that results ;^>

[2] https://en.wikipedia.org/wiki/Computus
I did not just make that up.

>>>>>>>>>>  - Date/time intervals/delta's.

Those I can see a better case for: they could be handled by operator-()
methods (with returns measured in the given units) of QDate (days),
QTime (milliseconds), QDateTime (milliseconds, 64-bit).  No operator+;
use addDays(), addSeconds(), &c.

>>>>>>>>>> (by date/time deltas, I e.g. I started working somewhere on
>>>>>>>>>> november 1st 2014. and stopped january 3rd 2015. How long did
>>>>>>>>>> I work there)

Quite.  Note, however, that KCalendarSystem::dateDifference() raises a
valid point for broken-down difference in days, months, years - which
need not equate to a simple number of days. [2016-03-01] - [2016-02-01]
is 29 days but the dateDifference is 1 month; while the same in 2017 is
28 days but still 1 month; and [2017-02-01] - [2016-02-01] is one year,
the same as [2017-05-01] - [2016-05-01], but the former is 366 days and
the latter is 365 days.  Other calendar systems shall doubtless give
more such subtleties.

André Somers
>>>>>>>>> If memory serves me right: When, years ago, I tried to get the
>>>>>>>>> latter in, the work was a bit blocked because somebody else
>>>>>>>>> what working on... calendar support. :-)

If you can find either your patches for diff or their patches for
calendar support, I'm sure they'd be useful input to the present effort.

Ch'Gans (a.k.a. Chris):
>>>>>>>> Boost have them all: date/time, calendar, time zone, time
>>>>>>>> period and time duration
>>>>>>>> Date/Time is a very tricky subject, why not rely on boost
>>>>>>>> implementation?
>>>>>>>>
>>>>>>>> http://www.boost.org/doc/libs/1_51_0/libs/locale/doc/html/group__date__time.html

The documentation is a bit thin - leaving me guessing at rather more
than I should need to - but my impression is that this isn't actually as
good as what we have, although it does have some (inadequately
described) handling of alternative calendars.  I'm fairly sure we can do
better.

Thiago
>>>>>>> I don't expect the calendaring system to require any changes to
>>>>>>> QDate internals. It stores a Julian day, that's all.

The QDate data object's internals won't need to change, no; but the code
it presently uses to map between its internal Julian day and years,
months, etc. is all Gregorian and would need rearranged.

Soroush:
>>>>>> QDate knows how many days are in a month, what are month names
>>>>>> (in case locale is not present), how to find leap years, and how
>>>>>> to convert a JulianDay to year, month and day in Gregorian
>>>>>> calendar.

... all of which should clearly move to the Gregorian calendar class
that'll still be used by default to do these jobs; but we can have
methods accept another calendar object instead.

Thiago:
>>>>> QDate will continue to support Gregorian day, month, year, as well
>>>>> as ISO weeks.

... *by default* - but we should be able to have it support some other
calendar's day, month, year and weeks by supplying other calendar
objects to suitable overloads of methods.

Frédéric Marchal:
>>>> As a developer, I don't want to know about the gory details of date
>>>> computation with calendars I have never heard of. Adding "one
>>>> month" to a date should use the same code whatever the underlying
>>>> calendar is.

The calendar wouldn't be "underlying" in what I'm suggesting: client
code (e.g. a calendar app) would need a calendar-system object to pass
in to QDate's methods.  We can't have QDate remember which calendar to
use (ABI change); in any case, I'm fairly sure I don't want it to - as I
do also want to be able to ask about the Gregorian date of the start of
Ramadan (for example), which will mean creating a QDate with one
calendar system and using another to query it.  The QDate shall remain
just a Julian day packaged with some suitable methods, some of which
shall know how to interact with calendar systems.

>>>> The correct calendar to use should be a simple configuration
>>>> parameter

How the app built on Qt picks its calendar system is for the app to
decide.  If the app has a config enum that tells it which
Q*CalendarSystem to instantiate, that's fine by me: but I do also want
apps to be able to support calendar systems that aren't mass-market
enough that it makes sense for *us* to support them.

The nice thing about an instance-based approach is that an app can even
support loading a user-selected plugin that supports an API that lets
the app extract a calendar-system object that describes the calendar the
user actually wants to use - all without the app developer (or us)
having to know the details of that calendar system, merely how to load
the object for it.  Then the long tail of users with arcane calendars
can be catered to without us (or the app developer) having to ever see
(much less write, maintain or debug) the code that implements it.  One
member of the community that uses that calendar system shall need to
write the plugin (or whatever) and then all apps that know how to load
that plugin can support those users to their satisfaction.  We could
just support the more widely-used calendar systems.

Soroush:
>>>> Can we add arguments to all methods with a default value? Something
>>>> like:
>>>>
>>>>     QDate d;
>>>>     qDebug() << d.year();               // prints 2016
>>>>     qDebug() << d.year(QCalendar::Jalali); // prints 1395

That's what I have in mind, only with a QJalaliCalendar() instance in
place of your enum.

>>>> And then force relevant widgets/views to show/edit date and time on
>>>> a specific calendar system:
>>>>
>>>>     QDateEdit de;
>>>>     de.setDate(d);
>>>>     de.setCalendarSystem(QCalendar::Hebrew); // Is this possible?

That's trickier, as it would change QDateEdit's ABI.  Changing QDateEdit
to support other calendar systems would likely involve a major re-write.
However, a calendar-system-aware QDate might help you write your own
CalendarAwareDateEdit widget, based on QDateEdit.  It might then replace
QDateEdit in Qt6.

Lars Knoll:
>>>> I wonder whether we can't keep handling of different calendars
>>>> completely outside of QDate. Something similar to what we've done
>>>> with QString/QLocale. So QDate would continue unchanged and only
>>>> support the standard Gregorian calendar. In addition, we have a
>>>> QCalendar class, that can be constructed with a different calendar
>>>> system, and can then return 'localized' date strings, days, months
>>>> and years for this calendar system.

That would be another option (and it would approximately just upstream
KCalendarSystem); but I actually think the plan I sketch above can work
and will be more natural.  When it comes down to it, QDate really is
just a Julian day number and everything it does for the Gregorian
calendar is well-enough separated that I believe we can split it out.

>>>> Something like:
>>>>
>>>> QDate date;
>>>> QCalendar c(QCalendar::Hebrew);
>>>> QString hebrewDateString = c.toString(date);
>>>> int hebrewYear = c.year(date);

What I have in mind would look like: <code>

Type func(const QDate &date, const QCalendarSystem &calendar)
{
    QString text = date.toString(calendar);
    int year = date.year(calendar);
    ...
}

Type val = func(QDate(QJalaliCalendar(), 1395, 7, 15), QHebrewCalendar());

</code> (and I have no idea what I just did), except that the naked
instantiations would, in real application code, almost always be
replaced by forwarding const QCalendarSystem & values obtained from
elsewhere.

A calendar app can then have several panels or views, each with an
independent choice of calendar system; it the user drags a date from one
to another, it's just a Julian day so transfers safely but gets
displayed utterly differently after the transfer; all the same, the
events on the day get carried over just fine.  Or I can drag an event
from one to another; the event remembers its day, so lands in the right
place, for all that this is displayed differently.  The user can change
the calendar system in use by a view; all the entries change their
appearance but the underlying QDate objects stay the same.

>>>> Maybe one could even integrate this into QLocale, that already
>>>> provides support for localized month and day names?

QCalendarSystem or the QDate code using it is clearly going to need to
interact with QLocale, if only to get localised names of months and
week-days.  That probably won't happen by translating "February" or
"Wednesday" though; there may be no clean match between the months and
days of the other calendar and those of the European calendars.

However, calendar system is separate from locale; see the last paragraph
of my earlier mail and Soroush's response to it.

Frédéric Marchal
>>> There is more to it than converting a date to a string:
>>>
>>> * Add N days to a date.

That one's easy, because QDate is just a Julian day; increment it by N.
The part needing work is addMonths(const QCalendarSystem&, int) and
addYears(const QCalendarSystem &, int).  Eminently feasible.

>>> * Find the number of days in a month.

Not something we presently do for Gregorian; but not an unreasonable
thing to consider adding to the QDate (and QDateTime) API:

  int daysInMonth(const QCalendarSystem &cs = QGregorianCalendarSystem());

>>> * Compare two dates.

No need to change, just compare Julian day numbers.

>>> * Count the number of days between two dates.

Difference between Julian day numbers, easy.  More fun, but eminently
do-able, is the equivalent of KCalendarSystem::dateDifference().

>>> For instance a program wishing a happy new year to its users should
>>> do it with as little modifications as possible.

Yes; and I think that's feasible.

>>> Using a plain QDate would have been the easiest way to reach more
>>> users because it doesn't require to replace lots of QDate with a
>>> new, very similar, class. As it is not possible to change QDate for
>>> now, Soroush is looking for a temporary solution that would bridge
>>> the gap until Qt6 is out.

I think what I have in mind will manage this without needing to wait for
Qt6 (although we might want to clean up at Qt6).

Lars Knoll:
>> it might be a good idea to have this functionality in a class
>> separate from QDate. We’ve done the same design decision for QString
>> (having no locale specific functionality in QString), and this worked
>> out rather nicely. So I would encourage you to have a look whether
>> and how a similar design could be done for calendar system support.

QDate already has (Gregorian-based) methods entangling it with notions
of year, month, week and day within each; this is intrinsically
calendar-system-specific.  Aside from that it's just a Julian day,
stored in an oversized data-type (qint64, where int would suffice); so
there's nothing there except the bits that we'd need to remove to do the
equivalent of taking QLocale out of QString.

Sérgio Martins very helpfully linked to KCalendarSystem - thank you.
One of the things we should clearly aim for is to make it easy for
KCalendarSystem to (when its developers care to and can find the time,
with no pressure to do so) adapt to use QCalendarSystem and the adapted
QDate.  TODO: I also need to find and talk to its maintainers.

Thiago:
> Just like QString being able to format numbers in one particular
> locale (the "C" locale), we ought to support the C locale's
> calendaring system in QDate too.

I entirely agree that we should continue to support Gregorian by
default.  I think it makes sense, though, to have QDate work with other
calendaring systems.  Even if QCalendarSystem sits outside QDate and
does all the work for itself, then - with a putative

  QGregorianCalendarSystem greg;
  QDate date;

in scope - date.method(args) shall in many cases be equivalent to
greg.method(date.toJulianDay(), args) and QDate(args) to
QDate::fromJulianDay(greg.julianDay(args)).  It seems needlessly
limiting, to support for other calendar systems, not to also let these
be expressed in QDate as date.method(greg, args) and QDate(greg, args),
respectively, so that we can replace greg with another calendar system
object and save clients some tedious boilerplate.

I think my main reason for wanting this is that, without it, code that
supports non-Gregorian calendars shall have essentially no use for QDate
- its only use would be to extract its .toJulianDate(), do real work
with that, then QDate::fromJulianDate() back in order to interact with
the rest of Qt.  It would then make more sense for such code so simply
discard QDate and just use a Julian day instead - the need to convert
to/from QDate in order to interact with Qt would just be a hindrance,
rather than a help.

QDate should make it easy for app developers to add support for other
calendar systems, with minimal re-writing, rather than seducing them
into writing Gregorian-specific code that they would have to throw away
entirely in order to add that support.

... and that's me caught up with the thread.

Issue: Q(Date|Time)+ think day-changes happen at midnight.  Some
calendar systems think they happen at sunset or sunrise; these are both
rather tricky, as their time depends on date and latitude [3] - and I'm
not sure what they do about days when the sun doesn't set or rise at
all.  I don't see any hint of coping with this in KCalendarSystem and
I'm fairly sure I don't want to solve that problem, so I think we stick
with midnight and leave apps using the result to fix up after the fact
if they really care.  Indeed, what *is* common practice in The Middle
East, as regards how evenings get handled - as the tail of one day or
the start of the next ?  Has secular society shifted the day boundary to
midnight in practice ?

[3] also geography, altitude and even local climactic complications:
    https://en.wikipedia.org/wiki/Novaya_Zemlya_effect

Kindred issue with the ancient Julian calendar: before some time
mediaeval, when year transitions got moved to a month boundary, the day
after (somewhen like) March 14th of one year would be March 15th of the
*next* year.  (March is month 1: that's why Sept = 7, Oct = 8, Nov = 9
and Dec = 10 are the embers of the year, whose month numbers are now two
more than what their names claim.)

That one probably can be handled, but it's the sort of mess I'd rather
not implement myself, so I want an API that lets the app, or a plugin
it's loaded, supply the calendar system object to handle it.  Some
calendar systems matter a lot to those that care about them but are
nowhere near mass-market enough that it makes any sense for us to plan
on implementing them all.  There are a few widely-used calendars and a
long tail of more obscure ones: we need an architecture which lets
client code cater to the ones that we aren't about to handle natively.

KCalendarSystem's daysIn{Year,Month,Week} are sensible methods for the
calendar system class, albeit taking a Julian day rather than a QDate
(which is equivalent).  Various other methods that take or return QDate
would replace it with a Julian day; QDate would then use these in place
of its present hard-coded Gregorian code.

	Eddy.