Question: Edges of different types

[Jones-Adam]

Hi,

I have managed to compile and get the friends example working, so I think I have more or less understood how to represent 1 to many relationship in TSL. However its not entirely clear for me how to represent the edges where I have a single node type which can have 5 different edge types connecting the nodes. Both the nodes and the edges hold data.

So e.g. I can have

cell struct myNode
{
    string name;
    string owner;
    list<long> edges;
}

cell struct myEdge
{
    long StartNode;
    long EndNode;
    int8 EdgeType;
    float weight;
    string sourcePath;
}

which lets me find the start and end myNodes given any myEdge and lets me get the list of myEdges for a given myNode. However its less friendly to then get from a given myNode only those myEdges which have a certain edgeType.

Is there a better way to structure this to make this kind of query more direct? Not every myNode has every edge type and there may be multiple myEdges of a given type connected between two myNodes

Thanks in advance for any help you can offer.

[TaviTruman]

Hello Adam Jones (@Jones-Adam) - the Graph Engine can easily and quite cleanly support Directed Graph and even HyperGraphs productions in TSL; 1-N, N-N, Fully Optional and other cardinal contraint typicalities are all possible. You should use the TSL "Attribute" production to tell the TSL compiler to generate metadata and meta-processing (foreach over your attributeKey) that you can use in your GE LINQ code.

cell struct myNode
{
string name;
string owner;
[GraphEdge:MyEdges]
List edges;
}

cell struct myEdge
{
CellId StartNode;
CellId EndNode;
[EdgeTypeDescriptor:MyEdgeType]
int8 EdgeType;
float weight;
string sourcePath;
}

Take a look at the generated code produced by the TSL compiler and you can see that it is quite easy to compose very clearn code that meets your needs:

You an use GE generarted Accessor and Selector to write imparative and or LINQ Functional code to query all edgeNodes that conatain myEdge where EdgeType = is a value in set of edge types.

Take a look at this documentation if you have not already done so:

https://www.graphengine.io/docs/manual/TSL/tsl-basics.html
https://www.graphengine.io/docs/manual/TSL/accessor.html

I'd also suggest that you take a look at the "Freebase" sample project and look ove the "freebase.tsl" file. The Freebase samples use LIKQ but the TSL tells important design concepts.

[Jones-Adam]

Thanks for the answer

I have read through both those articles and tried to work out what is going on with the freebase example and I have added in the attributes as suggested but I'm afraid I am still missing something somewhere.
To look up the edge I want to use something like:

var node = Global.LocalStorage.MyNode_Selector().Where(p => p.Name.StartsWith("xx_99_01")).First();
 using (var cm = Global.LocalStorage.UseMyNode(node.CellId))
 {
       var type1Edges = cm.Edges.Where(p => ... );
 }

I don't see what to put in the .... The p refers to long in the list of edges and not the edge itself. and since I cannot nest accessors I don't quite see how get back a list of edges filter on EdgeType. The accessor collection

Alternatively I suppose I can do

var node = Global.LocalStorage.MyNode_Selector().Where(p => p.Name.StartsWith("xx_99_01")).First();
var type1Edges = Global.LocalStorage.MyEdge_Accessor_Selector().Where(p => p.StartNode == node.CellId && p.EdgeType == 1);

However if I understand correctly this enumerates through the whole list of edges, something I don't think I really want to do as there will be several million such edges once I load all the data. It seems wrong to do that especially if I already have a reference to a starting node. Or am I over complicating things and the library handles this behind the scenes?

Just incase it makes a difference, the only thing I can see different from what you said and I have done is that in the TSL file I use in the definition of edge List as List by itself won't compile...

[TaviTruman]

Adam Jones (@Jones-Adam) When you used the TSL "Attribute" language feature type of reservation made for you so that you can write your custom indexing, search and any type of relational processing. I have been writing an OWL 2 DL/Full Semantic Reasoner for the past year that is built on top of the GE and it's graph implementation is based on an RDF-HyperGraph model so, I have learned a lot about what is possible.

The code that you want to study is in the LIKQ solution and the FanoutSearch implementation. I'm going to explain some of the theory and that's behind what the Trinity team refers to as "Graph Exploration." When I first got started I ran into some of the same type of head-scratching you're going through now. There's a number of research white papers that describe the foundational theory and algorithm that drives the "Selector" and "Accessor" capabilities.

It is not documented strongly but you can implement Message, Selector and Accessor Handlers/Interceptors and as we are dealing with "Directed Graph" you going to want to deal directly with "Node", "Edge" and "EdgeType" processing and that mean you need to work with "ICellAccessor."

When you specify the TSL Attribute [GraphEdge] know and understand that it has special first-class meaning in the computed model; here is an example of how you can leverage TSL Attributes:

Graph Engine Demos:

I'm staniding ujp a GitHub site that will have a bunch of demos; that seems to be a better place to address more complex topic. The stock supplied demos are a great place to start but we need more. Graph data systems and databases are very interesting, cool and fun but more importantly, you can do amazing things with them. I'll come back later and update with a link to the Graph Engine Demo site.

[Jones-Adam]

I am very much looking forward to your promised demo, What I would like to do is clearly far more involved then I was hoping it would be, but learning is fun. In fact the more I am learning about the world of directed hypergraphs the more interested I am in having a "database" to store them in.

[TaviTruman]

Hey, Adam Jones (@Jones-Adam) I am dealing with health issues at present due to the fire we've had here in SF Bay Area; however, I will upload the demo project in the next day or so. I'll come back here and update with a link to where I am posting a lot of demos regarding Graph Engine, RDF-Directed Graph and RDF-Hypergraphs with the Graph Engine.

[Jones-Adam]

Tavi Truman (@TaviTruman) oh sorry to hear that, hope you feel better soon.

[TaviTruman]

Feeling better today. I'll be working to complete a few demos and upload today. Thanks for the well wishes.

[TaviTruman]

Adam Jones (@Jones-Adam) Hey there - I'm back and feeling much better. I thought I'd back up a bit before getting into the deeper aspect of TSL Data modeling with the Graph Engine, RDF-Graph, Property Graph (Labeled Graphs) and Hypergraph schemas. We want to look at many-2-many relationships of vertices and edges in the graphs. If you have been reading around the internet this is a long-talked-about and interesting topic so I want to make certain that get solid understanding in place w.r.t. Relational Modeling with TSL. I'm building the GitHub website resource tonight and I'll be working on dos, presentations, and demos in a concentrated way. I'll come back here over the next few days with updates and demos will be available this week.

[Jones-Adam]

Tavi Truman (@TaviTruman) I found the GitHub site and the readme looks promising :) I can imagine the run up to the end of the year is as busy for you as it is for me, My original use-case I have decided to handle in a different way temporarily until I have the time to get my head around this topic better. I have also been trawling through the source code of HyperGraphDB (unfortunately in java) which is perhaps more aligned to what I was originally thinking of an API than how GraphEngine is constructed. I read through some of the forum comments and presentations on why GraphEngine is the way it is. I think the fundamental difference in my use-cases vs what is done in the Freebase demo is that with Freebase the structure of the data is known in advance, while I have a structure which can evolve at runtime with new node or edge types being added/modified by end-users/other systems - hence the hypergraph approach, but perhaps another way more in keeping with the GraphEngine philosophy would be to dynamically extend the graph structure at runtime (vs TSL generation) with the modified types. This would then require some kind of schema sync between the distributed nodes and perhaps a basic data migration path or at least default value handling ... At this point, which road would be the more complicated/advantageous is not clear to me, perhaps once I can see your hypergraph schema demo it will be obvious :)

[TaviTruman]

Adam Jones (@Jones-Adam) I'm just getting to my desk after being sick with an upper respiratory infection. I'm in the process of updating my new GitHub Repository over the next couple of days. Keep an eye out for updates.

[TaviTruman]

Adam Jones (@Jones-Adam) I'm going to suggest we close the discussion here. I am working one another github repository to address these type of questions; I'll share the lessons I've learned in working the Graph Engine as well as address a bunch of relatred topics I'm very excited about. The Graph Engine is super cool and very powerful but does take a little while to warm up to; it a new type of design and programming model and a bit more advanced skill is required to really make it wiggle. I think there is a lot room to democratize the technology and make it more approachable to devs. I'm in the mist of heavy product development using the GE and am finding it very hard to make time to published works but it it is coming along.