You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the process of working on a new repo driver, I’ve run up against some of the limits of the ReadRepo interface. Namely:
Each implementation of it so far in eventhorizon has its own ad-hoc methods. For example, the Mongo driver has FindCustom and FindCustomIter, which take Mongo-specific arguments. The problems with this are two-fold:
It’s difficult to swap out implementations, which makes testing hard (e.g. if we don't have access to a real database or just don't want to spin one up for speed reasons and would rather use local).
The acceptance tests don’t cover a large portion of the ReadRepo functionality, which makes it hard to implement new drivers that follow the conventions of existing drivers and are tested with the same level of rigor.
Find and FindAll alone don’t fulfill the query needs of a fully-featured application. It’s missing paging (a must-have for a large number of records), selecting by any kind of condition, making aggregation queries (e.g. count of records which satisfy a condition), sorting, and probably some other things I'm not thinking of. The MongoDB repo solves this in an ad-hoc way by exposing the ability to execute MongoDB queries directly. Also, from what I can surmise the Parent() method on ReadRepo exists only to access this MongoDB-specific functionality at the moment.
The interface itself, with its Find and FindAll methods, seems more informed by MongoDB than CQRS. This is a completely semantic argument, but to me naming something after the MongoDB API makes it marginally more difficult to map the functionality to CQRS principles.
Based on these limitations, I would like to propose the following replacement for ReadRepo:
// Querier exposes an interface for retrieiving Entities based on a query.typeQuerierinterface {
Query(ctx context.Context, opts...QueryOption) (*QueryResults, error)
}
// QueryOption modifies a query to return limited results based on some// restriction. A query may take multiple QueryOptions or none.typeQueryOptioninterface{}
// QueryResults represents the result of a query.typeQueryResultsstruct {
Entities []Entity// One page of entities matching the query.Totaluint64// The total number of entites matching the query.
}
// Possible general QueryOptions.// NewQueryOptionEntityID returns a QueryOption which restricts the results// of Query to return only a record matching the id (if it exists).funcNewQueryOptionEntityID(idUUID) QueryOption {
returnQueryOptionEntityID{id}
}
typeQueryOptionEntityIDstruct{
idUUID
}
// NewQueryOptionSearchTerm returns a QueryOption which when passed to Query// will perform a full text search over the specified fields using term as// the search term.funcNewQueryOptionSearchTerm(termstring, fields []string) QueryOption {
returnQueryOptionSearchBy{term}
}
typeQueryOptionSearchTermstruct{
termstringfields []string
}
// NewQueryOptionFieldExactMatch returns a QueryOption which restricts query results// to ones for which the value of the specified field matches exactly one of the// specified values.//// E.g., A query over the Entity collection// [// {"firstName": "Jane", "lastName": "Orman"},// {"firstName": "Lily", "lastName": "Marlowe"}// ]// with this option with field set to "firstName" and values set to ["Jane"] should// return// [{"firstName": "Jane"}]funcNewQueryOptionFieldExactMatch(fieldstring, values []string) QueryOption {
returnQueryOptionFieldExactMatch{field, matches}
}
typeQueryOptionFieldExactMatchstruct{
fieldstringmatches []string
}
// NewQueryOptionPage returns a QueryOption which specifies what page of// Entites to retrieve.funcNewQueryOptionPage(limit, pageuint64) QueryOption {
return!ueryOptionPage{limit, page}
}
typeQueryOptionPagestruct{
limit, pageuint64
}
// NewQueryOptionPage returns a QueryOption which specifies how the results will// be sorted.funcNewQueryOptionSort(fieldstring) QueryOption {
returnQueryOptionSort{field}
}
typeQueryOptionSortstruct{
fieldstring
}
And then an example of how an implementation of Querier might handle QueryOptions:
func (m*MyQuerier) Query(ctx context.Context, opts...eh.QueryOption) (*eh.QueryResults, error) {
varlimituint64varpageuint64varwherestringfor_, opt:=rangeopts {
switcht:=opt.(type) {
case eh.QueryOptionPageLimit: // Example of built-in QueryOption.limit=t.limitpage=t.pagecaseMyWhereClause: // Example of custom QueryOption.where=t.wheredefault:
returnnil, fmt.Errorf("unsupported type: %T", opt.(type))
}
}
// Use DB-specific logic along with the variables declared at the top// of this function to make a query.// ...return&eh.QueryResults{entities, total}, nil
}
A few notes about this:
The QueryOption idea is heavily inspired by gRPC DialOptions, which are also passed as variadic arguments.
The interface (in query.go, perhaps) can expose some general QueryOptions itself. The implementations of Querier can choose to support them or not, and the implementations may also expose their own QueryOptions. This way, implementations of Querier may be extensible without breaking the interface API, so we can write tests for the way our transport layer (e.g., RPC or HTTP) uses the Querier using a mocked interface. As an example, we may achieve backwards compatibility (probably) with any applications using the MongoDB driver with FindCustom currently by exposing a queryOptionMongo that contains a func(*mgo.Collection) *mgo.Query, which is what FindCustom does anyway.
There is no method exposed for accessing a single record (i.e. Find is gone), but combining NewQueryOptionID with a check for an empty slice of entities returned from Query can achieve the same functionality. This keeps the interface very simple.
Paging is a first-class citizen here since as I was mentioning above, it’s really a must have for any large dataset.
This doesn't solve at all the problem I mentioned about aggregate queries (like SQL's group by), but potentially that could be solved by a separate Aggregator interface. That also may just need to be very custom.
This is all just stuff I’ve thought about in the last two days or so, and I am very open to any and all feedback. Everything I’m trying to do here is in service of making it easier to work within the Event Horizon framework so that when using it we’re thinking more about how to do CQRS well rather than how to do something or other with any specific database, so I’m especially interested to hear suggestions about how we can get closer to that ideal.
Also thanks to @hryx, with whom I’m working on all of these ideas. Please chime in if I've missed anything.
The text was updated successfully, but these errors were encountered:
I've had some experience implementing some Query engines and I came up with the following interfaces:
// Query defines query parameters.typeQueryinterface {
QueryID() string
}
// QueryHandler handles a query.typeQueryHandlerinterface {
// Handle handles the given query and returns the report.Handle(ctx context.Context, qQuery, reportinterface{}) error
}
The typical usage example may look as follows:
// querytypeGetAccountShortInfostruct {
Numberstring
}
func (rGetAccountShortInfo) QueryID() string {
return"GetAccountShortInfo"
}
// report (read side model)// Account An Account representation.typeAccountstruct {
NumberstringBalance money.MoneyLedgers []Ledger
}
// somewhere in the application service where results are neededfuncShowAccount {
accountReport:=&report.Account{}
err:=h.queryBus.Handle(context.Background(), query.GetAccountShortInfo{Number: number}, accountReport)
// accountReport will have all the query results
}
Obviously the query structure can hold all the fields needed for the request: limit, filters, offset, etc...
You may for example implement some CriteriaQuery and an SQLQueryHandler which would satisfy the interfaces above :)
Follow-up of this Slack conversation.
In the process of working on a new repo driver, I’ve run up against some of the limits of the
ReadRepo
interface. Namely:eventhorizon
has its own ad-hoc methods. For example, the Mongo driver hasFindCustom
andFindCustomIter
, which take Mongo-specific arguments. The problems with this are two-fold:local
).ReadRepo
functionality, which makes it hard to implement new drivers that follow the conventions of existing drivers and are tested with the same level of rigor.Find
andFindAll
alone don’t fulfill the query needs of a fully-featured application. It’s missing paging (a must-have for a large number of records), selecting by any kind of condition, making aggregation queries (e.g. count of records which satisfy a condition), sorting, and probably some other things I'm not thinking of. The MongoDB repo solves this in an ad-hoc way by exposing the ability to execute MongoDB queries directly. Also, from what I can surmise theParent()
method onReadRepo
exists only to access this MongoDB-specific functionality at the moment.Find
andFindAll
methods, seems more informed by MongoDB than CQRS. This is a completely semantic argument, but to me naming something after the MongoDB API makes it marginally more difficult to map the functionality to CQRS principles.Based on these limitations, I would like to propose the following replacement for
ReadRepo
:And then an example of how an implementation of
Querier
might handleQueryOption
s:A few notes about this:
QueryOption
idea is heavily inspired by gRPCDialOption
s, which are also passed as variadic arguments.query.go
, perhaps) can expose some generalQueryOption
s itself. The implementations ofQuerier
can choose to support them or not, and the implementations may also expose their ownQueryOption
s. This way, implementations ofQuerier
may be extensible without breaking the interface API, so we can write tests for the way our transport layer (e.g., RPC or HTTP) uses theQuerier
using a mocked interface. As an example, we may achieve backwards compatibility (probably) with any applications using the MongoDB driver withFindCustom
currently by exposing aqueryOptionMongo
that contains afunc(*mgo.Collection) *mgo.Query
, which is whatFindCustom
does anyway.Find
is gone), but combiningNewQueryOptionID
with a check for an empty slice of entities returned fromQuery
can achieve the same functionality. This keeps the interface very simple.group by
), but potentially that could be solved by a separateAggregator
interface. That also may just need to be very custom.This is all just stuff I’ve thought about in the last two days or so, and I am very open to any and all feedback. Everything I’m trying to do here is in service of making it easier to work within the Event Horizon framework so that when using it we’re thinking more about how to do CQRS well rather than how to do something or other with any specific database, so I’m especially interested to hear suggestions about how we can get closer to that ideal.
Also thanks to @hryx, with whom I’m working on all of these ideas. Please chime in if I've missed anything.
The text was updated successfully, but these errors were encountered: