logs archiveChat DB / Freenode / #programming / 2015 / November / 5 / 6
pyon
jeaye: Ah, yes, Clojure has better data structures in the standard library.
But what I need is for the most part macros. w.r.t. macros, Racket is better.
jeaye
So I've heard.
oded
pyon: what do you call structured data? the fact that the schema is predefined? you can do that when using NoSQL as well, or is it something else?
pluszak
gosh, why american IT people use mmddyyyy? I understand uneducated average joe's but guys in the trade?
pyon
oded: What I mean is that the schema is *enforced*.
nso95_
pyon: what is all this code you're always writing?
pyon
oded: And has mathematical laws that help me quickly determine (by means of either calculations or mathematical intuition) what kind of queries makes sense.
oded
well you can enforce a schema in MongoDB etc, it's a matter of the application
pyon
oded: You think I'm going to trust *JavaScript* with enforcing anything?
oded
pluszak: it's the whole country
pluszak
oded: but you'd assume that guys that deal with IT would see it's absurd
pyon
oded: To a Haskeller/MLer like me, enforcing has a much more precise meaning than perform a bunch of runtime checks.
adsc
pluszak: they are probably just used to it
oded
pyon: I mean, you can develop an infrastructure or use an existing one that enforces. What makes MySQL a better enforcer other than the fact that they *decided* to not build collections on INSERT time
pyon
oded: I like how SQL lets me know things in advance about my data, without having to make a single query. For instance, I can tell that, for every Foo, there is exactly one Bar, and the only way to retrieve it is to get the Foo's Qux, and then the Qux's Bar.
oded
pluszak: I'd expect anyone to see it's absurd... because it's absurd
pyon
oded: I don't think MySQL is a terribly good relational database.
database management* system*
adsc
but but but it's may the third not third the may
oded
nevermind MySQL specifically, ok what you're saying is that you like that relational databases enforce the relationships
pyon
oded: Yes. Relational databases understand and enforce the structure of your data. You can do better than relational databases, but NoSQL goes in the opposite direction of the correct one.
funrep_
is it possible to use Java JAXB without Xmlschema?
koollman
I'm fairly sure there is many definition of 'correct'
funrep_
because i have a class already for what i want the xml data to "get into"
koollman
*are
pyon
koollman: Well, correct w.r.t. my goal of statically enforcing more invariants.
Every statically enforced invariant is something you don't need to test.
Because, um, it's already enforced.
koollman
yeah, definitely not correct in this context :)
oded
pyon: I get what you're saying when your problem is relationships. What I wrote is that you should only use a relational database for relationships. But documents are not relationships, and keeping them in a relational database creates load on the wrong places
pyon
oded: Sure.
oded: I'm not saying store your JSON in a relational database.
oded: What I'm saying is figure out the actual structure of your data, so that you don't need to use JSON to begin with.
We have a whole generation of programmers who have no frigging idea what a data structure is.
oded
pyon: Say a Word document. You can split the words and store them in a words table with position. Is that what you mean?
knight-
i've been using relational databases my whole life... would love to find a NoSQL-like database that makes sense... so many of them are far too right, and SQL is often far too left... there's not a lot in the middle.
pyon
oded: I wouldn't store Word documents in a database.
oded
knight-: try this http://programwithhead.blogspot.co.il/2015/11/sql-vs-nosql-vs-search-engine.html let me know what you think
pyon
oded: I'd store actual structured data. When a user needs a Word document, the program fetches the necessary data from the database, and builds the Word document on demand.
oded
well define the word database
:)
pyon: lets call it a data store (I had that problem with the wording in writing the article actually)
what I tried to say there is that like you said, DON'T store structured data (data with relationships) in NoSQL because "it's easy", store it in SQL
pyon
oded: I'd define a database as a *consistent* collection of facts about a domain, that satisfy a bunch of first-order predicates.
oded: First-order as in first-order logic.
knight-
i mean, i often have data that needs to be structurally dynamic... not necessarily optional, but changes based on user or organizational input... document based databases make the most sense for this, but you lose a lot of compatibility with frameworks and search/query-ability when you do so. and relational databases often don't make sense for this type of data because you lose the ability to have referential integrity and indexing and a lot of
other things.
there really isn't a happy middle ground... there's nothing inbetween SQL and NoSQL... hybrids are *starting* to pop up... for example, PostgreSQL has been support JSON columns for a while now, but compatibility with it is rather lacking in most places... you also lose standard query syntax, and it's not as easy to manipulate.
oded
knight-: I disagree on that, by saying "it changes based on user or organizational input" means you didn't find the correct generic structure, or you have more than one structure on your hand (two different unrelated applications?)
knight-
no, not true. a generic structure would be so generic that you lose the performance benefits, indexing, and integrity that the database offers.
for example, i have created a recommendation engine that takes in dynamic attributes from an organization, which are manipulated and weighted. for now in SQL they are individually entered as rows rather than structure, allow for an unlimited number of attributes... but it forces them to be "virtual" attributes, where the types are defined as metadata, and then functionality built around in code to enforce the functionality. the database doesn't
get to enforce that a string attribute is a string in this scenario.
oded
knight-: usually after you separate your problem into smaller blocks you will find that multiple generic solutions solve specific problems
knight-
yep, it's totally capable of being generic (it is now)... but you lose the added benefits that having things defined deeper in the structure provides by the database
such as linking ids for referential integrity
oded
knight-: in that situation I'd recommend MTI, I'm using it now actually on something similar to what you said
pyon
knight-: If you need an unbounded/unlimited number of attributes, you really aren't doing relational design.
knight-
when you have crafted structure as *data* in a generic structure that wraps around it, then you lose the opportunity for the structure hiding in the data to be exposed as core database features
yep exactly
which is why it makes sense for this stuff to be in NoSQL
but NoSQL presents its own challenges with this sort of data as well
and as i mentioned previously, you lose a lot of the familiar integration patterns that SQL often provides
it's just a lot of tradeoffs
funrep_
i added a class inside a class in java and now i cant access it?
oded
What I do is have the MTI base in SQL (with all the relations it needs, and everything relating to it), and the class's "children" in different NoSQL collections (one for string, one for integer etc)
knight-
there needs to be a happier middle ground... SQL databases that are smarter at handling documents, and document based databases that offer more common relational opportunities
funrep_
i want to do soemthing like this: myClass.MyOtherClass.element
pyon
My main beef with NoSQL is that (today's) non-relational databases don't have a mathematical model for reasoning about their structure.
knight-
personally, i use BOTH.
oded
funrep_: add public static
pyon
So it's a fu*king lot of trial and eror.
knight-
i use the SQL generic structures to capture data, then the data is processed and injected into a NoSQL database for representation.
pyon
error*
knight-
it's a little nasty, but it works
pyon: yeah
funrep_
still doesnt work
oded: should i send code?
knight-
another problem with NoSQL --- they all use a different "query language" to access
oded
funrep_: gist.github.com
funrep_
2~http://lpaste.net/6588884233299689472
knight-
like Mongo... JS and map/reduce... functional interfacing...
oded
of both definition and usage
pyon
knight-: I don't mind that. NoSQL is actually an umbrella term for so many things - it's to be expected that they will offer different interfaces.
knight-
yep, but some standardization would make switching between things a LOT more reasonable
funrep_
oded: customer.tileset
sorry for naming ive copied code from tutorials ;)
knight-
doesn't have to be ONE language for NoSQL... but a set of standards would go a long way
funrep_
oded: line 23 is the error
pyon
knight-: As long as it comes with a reasonable mathematical model, I can find my way. But I haven't seen anything convincing.
knight-
SQL92 has gone a long way for compatibility and growing communities about relational databasing
s/about/around
funrep_
it says the field isnt in Map, but it is
oded
funrep_: you mean 24?
pyon
With SQL, all I need to do is frigging relational algebra. Accounting for aggregate functions doesn't require wild extensions - an aggregate function is just a commutative monoid.
funrep_
oded: yes
oded
knight-: why does it have to be one of the two? why not use both each for its own need
knight-
certainly, that's true. but then you lose any sense of mapping the data between them.
it becomes a chore of your application(s), and that has its own disadvantages
also, complexity rises significantly
« prev 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 next »