The Open Group releases DCE 1.2.2 as LGPL'd Free Software

Posted 14 Jan 2005 at 17:05 UTC by lkcl Share This

The Open Group announced 12th January 2005 that they are releasing DCE/RPC 1.2.2 as a Free Software Project - under the LGPL.

FreeDCE is a closely related project, which is the DCE 1.1 "Reference Implementation" updated, autoconf'd, with an implementation of DCOM added.

There is a mirror at opendce.hands.com which runs rsync, ftp, and there is also a dce122.tar.bz2.torrent bittorrent running as well.

The release of DCE/RPC as Free Software is highly significant. Its history dates back to "The Unix Wars" at a time when businesses were dependent on the Unix Vendors for their mission-critical services, spending a fortune, and "The Unix Wars" put them at risk. Hence, a call for technology went out, via The Open Group, and technologies like X-Motif and DCE/RPC were the result.

DCE/RPC then began to compete with CORBA (more on that later) and in an attempt to stem the tide, a reference implementation - DCE 1.1 - was released under a BSD-like Free Software license, and then later, DCE 1.2.2 itself was released under an "Open Source" license - Open Source in the strictest sense of the word (as in NOT a Free Software License) because the license specified "Educational purposes" only. Hence, a number of Universities immediately spotted that they could use it...

It didn't matter: it was too little, too late: CORBA was released under the GPL by Olivetti Research Labs in 1996. But, the thing about CORBA is that it is not only an RPC system and runtime environment, but it is also contains an Object-Orientated framework: DCE/RPC 1.1 is just an RPC system and runtime environment. To get your Object-Orientated framework, you need DCOM: see OpenGroup Meetings.

Only recently in the past few years have we had, in FreeDCE, a Free Software DCOM implementation. It's ironic that the technology that made Microsoft Software Development so successful - COM/OLE - is only now really coming to light, at a time when their embracing and extending has them stepping up to the ring on SOAP and .NET

Yet even this is somewhat irrelevant, although related to, what has been released: DCE 1.2.2 contains DFS - the Distributed File System. And I'm going to be bold enough to claim that DFS is the ONLY Distributed File System in the world that actually does what it says on the tin: distribute files. Getting locking right is a bitch: it requires an absolute dead-accurate distributed and correct view of what the present time is on every machine: hence, in DCE 1.2.2, you have a time server.

There's also a need to store authentication and authorisation and other bits and pieces and the DCE 1.2.2 code contains "Cell Directory Services". Many people have seen this as a limitation, and so in 1999, The Open Group announced an initiative to back-end DCE into an LDAP server. Code has not been made publicly available: DCE/RPC is still owned by its Copyright holders and they have chosen to release ONLY DCE 1.2.2... :)

DCE/RPC is the most amazing piece of programming I have ever encountered: the uses to which it has been successfully put are just staggering and mind-blowing.

  • DCE/RPC is the technology that allows companies like IBM and Fujitsu to tender for £500m contracts.

  • DFS ran the file server for the 1996 Olympics, allowing Web mirror sites across the world to automatically pick up changes made by staff in several locations to edit the site files as the Olympics progressed (see here and here and in particular, here)

  • The UK's "National Insurance" Database is a DCE/RPC application which must now hold aroung a TERABYTE of information as it started out with around 400Gbytes.

  • Microsoft adopted DCE/RPC and created not only NT Domain Services but also used DCE/RPC as the basis for its entire administrative toolchain, and with a few notable exceptions (based on a case-by-case conversion to Active Directory of which Exchange 2003 is one such example) still is! Microsoft's DNS Server can be managed via a tool that has a DCE/RPC-based GUI client: a few simple functions into an IDL file, hit compile and wham, you automatically have a remote management tool.

And we - the Free Software Community - are just _sitting_ here twiddling our thumbs thinking what a great job we're doing providing businesses with software for them to use, by reinventing the wheel with things like D-Bus (which contains no IDL compiler and no networking capability) and Global File System (which is proprietary anyway, available from Redhat), and by having to reinvent or hand-build an RPC system: Bayonne, the GNU/Hurd, the KDE project - the list goes on.

DCE/RPC is _way_ out of our league, and we are truly privileged to have access to it. Let's be ready and willing to step up to the challenge!

P.S. did anyone know that there has existed a full and Free Software implementation of an X-500 Server? Only now are people endeavouring to make up for LDAP's shortcomings, ironically by adding exactly the things that were originally in X-500, from whence LDAP (the L means light-weight!) came!


Redhat's GFS _is_ available, posted 14 Jan 2005 at 22:05 UTC by lkcl » (Master)

someone kindly pointed out an error in the above article: Redhat's GFS source code is available, here

i don't grok "opengfs" which i am pretty confused about [its existence]

in case you were wondering about nss and pam plugins..., posted 14 Jan 2005 at 22:38 UTC by lkcl » (Master)

in the days before DCE was free software, someone wrote some plugins for nss and pam for DCE:

http://www.intranet.csupomona.edu/~henson/www/projects/

and a whole boat-load more. mirrored here.

Motive, posted 16 Jan 2005 at 01:11 UTC by chalst » (Master)

Luke: great news. Do you have any insights as to why the Open Group is doing this now?

Also, how does the functionality of this release compare to that of FreeDCE?

Open Group's remit hasn't changed, posted 16 Jan 2005 at 13:29 UTC by lkcl » (Master)

Hello, hello.

The Open Group's goals and remit essentially haven't changed: they released DCE 1.1, the reference implementation, _years_ ago under the OSF 1.0 "BSD-compatible" license.

btw - DCE 1.1 _is_ FreeDCE with autoconf and DCOM added and a few other bits and pieces, including an updated DCE threads library that _almost_ correctly emulates POSIX Draft 4 semantics on top of Linux Threads (including NTPL, now, thanks to Loic).

it's just that a middleware runtime environment and library and development environment isn't particularly useful if you don't realise what it can do nor have any applications on which to gear things.

DCE 1.1 is a bit like the supercharged Boss 12 litre V10 engine that Ford used for its 1994 Ford Cobra concept car - capable of maintaining wheelspin up to well over 110mph - IF you actually bother to put that engine INTO a car!!!

... and in the free software world, nobody has!

but anyway - i digress.

the reason i believe why it's been released as free software is because the free software world "caught up" with and began to "overlap" the role of The Open Group. it's just such a pity that IBM were still making enough money out of DCE 3.0 to not be able to justify its release at the time that TOG wanted to - early 2001.

So.

here's what each of the things have, and where they differ. i won't go into DCE 3.0 (which is IBM's proprietary code that replaces CDS with an LDAP backend).

* DCE 1.2.2. 3.5 million lines of code. LGPL license.

  • contains CDS
  • DFS
  • distributed time server essential for DFS to operate correctly.
  • compiler
  • endpoint mapper
  • example servers
  • enough documentation to BURY you and bring the download tarball up to 90mbytes - sufficiently complete for anyone to TOTALLY reimplement this stuff if they chose and had a few years of their life to spare
  • hooks for additional transports and authentication
  • a sort-of-really-half-badly-ripped-out kerberos authentication module (sort-of uses DES somewhere. sort-of)
  • assumption that a POSIX Draft 4 threading library is available.

* DCE 1.1 - reference implementation, 250,000 lines of code, OSF 1.0 BSD-compatible license.

  • contains compiler
  • endpoint mapper
  • one example server
  • hooks for additional transports and authentication
  • all authentication ripped out
  • no services but the endpoint mapper
  • assumption that a POSIX Draft 4 threading library is available.

* FreeDCE 1.1 250,000 + a few lines of code, OSF 1.0 BSD-compatible license plus GPL on the DCE Threads library (yes, i know).

  • has been GNU/Autoconf'd
  • use of yacc brought up-to-date with latest version of flex
  • contains compiler
  • endpoint mapper
  • one example server
  • DCOM added (amazingly, thanks to wez furlong)
  • hooks for additional transports and authentication made easier to understand by being placed behind a .so loadable interface.
  • GSS-API authentication module added (.so)
  • NTLMSSP (for NT domains 3.5 -> 4.0) authentication module added (.so)
  • "Named Pipes" "outsourcing" transport added for samba tng to be used as the ncacn_np transport, (for samba 3 compatibility, see recent bit of bun-fight on Samba mailing lists for Dec 2004, ignore the bun-fight and look for Anthony's namedpipes "outsourcing" patch)
  • no services but the endpoint mapper
  • and Loic has kept Jim Doyle's original semi-working (if you had Redhat 5.1!) DCEThreads Posix Draft 4 threading emulation library up-to-date and working to the point where it _is_ actually useable, on x86 and AMD64, even under a 2.6 kernel.

The importance of FreeDCE, even as it stands, is that it is the basis on which the development of NT-compatible services may be accelerated: Exchange 5 (and most of 2000), MS-SQL, NT 3.5-> 4.0 Domains (and most of 2000), the list goes on.

None of these projects have sponsorship.

The strategic value of not one of these projects is yet realised or yet recognised.

The differences between DCE 1.2.2 and FreeDCE are that DCE 1.2.2 has more services, but has had virtually no work done on it since it was last made available (remind me to tell you the "what do you want _that_ old code for" story some day :)

The diffs between DCE 1.2.2 and 1.1 are "subtractions".

The diffs between DCE 1.1 and FreeDCE are "additions".

The overlap between these two is negligable.

Therefore it is quite easy to merge FreeDCE back into DCE 1.2.2.

The only hard work is going to be Autoconf'ing DCE 1.2.2.

you can work out for yourself that the importance of DCE 1.2.2 is high: for a start, it provides the world's only _fully_ POSIX-compliant Distributed File System (including coping with distributed byte range locking!)

nobody in the world has yet come CLOSE to what DFS can do, nobody in the world has yet come close to reproducing the functionality even of DCE 1.1, let alone 1.2.2!

*grrrrrrrrrrr* i want to bash heads together!

Wine, posted 16 Jan 2005 at 13:31 UTC by lkcl » (Master)

... _by_ the way... :)

if you wanted Wine binary-compatibility of Win32 DCOM-enabled applications to work, FreeDCE is _ab_solutely essential - as is interoperability with samba [outsourcing of "Named Pipes"].

Thanks, posted 17 Jan 2005 at 14:51 UTC by chalst » (Master)

Thanks for the summary of the differences. Another question: are the FreeDCE folks actually interested in the task of merging up to DCE-1.2.2? There's no sign of it on the public mail archives (at sourceforge).

DCOM documentation found on Open Group's site - full spec, posted 18 Jan 2005 at 01:25 UTC by lkcl » (Master)

implementation (proprietary, for sale):

http://www.opengroup.org/comsource/

documentation (online, must agree to terms and conditions)

http://www.opengroup.org/pubs/catalog/ax01.htm

interest and interested, posted 18 Jan 2005 at 01:50 UTC by lkcl » (Master)

short answer: yes.

long answer:

wez's efforts in 2000 to provide, with no funding, the free software community with a DCOM developer environment, went largely unnoticed and was underappreciated (see summary of wez's position at http://www.winehq.org/hypermail/wine-devel/2005/01/0500.html)

my efforts to provide the free software community with MSRPC services sufficient for businesses to get off of absolute dependence on NT servers was noticed - mostly - but not financially compensated, and the resultant fall-out from that has had a significant detrimental effect on progress, for which i can only apologise as it was not my intention, but an emotional reaction.

luke howard's efforts of three years solid work to provide an Active Directory replacement were successful and made commercially available some time over two to three years ago. his work is not available as a free software project.

the samba team's efforts to reengineer both the MSRPC services i wrote and a suitable runtime and developer environment are ongoing and do not overlap with either the samba tng project nor the FreeDCE/DCE1,22/DCE1,1 project. nor the kerberos/MIT project, nor the openldap project, nor the heimdal project, and up until recently, not the Wine project nor the OpenChange.org project, either.

the wine team's efforts to reengineer DCOM are at present completely ignoring or dismissing the existance and capability of FreeDCE, even though there exists full, comprehensive and accessible documentation for DCOM from the same site that brought you FreeDCE: The Open Group (see links in reply above)

basically what i am saying is that the people who have worked on this (FreeDCE) have spent a lot of time and effort on it, and a lot of them have been drastically underappreciated and underutilised, for one reason or another.

wez is now working as a php programmer in the US;

from what i can gather, elrond has finished his studies and is now working part-time as a sysadmin.

i am nearing financial ruin due to someone ripping my business partner off last year for approximately £30 to £50,000.

if someone gets these people some money, that will go a long way towards ensuring that the people with skills and expertise regain the interest that they have [effectively] been encouraged to lose.

i'm really sorry that that sounds really bad, but it's what i understand to be the truth. i have a habit of answering the question that is asked, and find it extremely difficult to deliberately withold information or deliberately lie, even though the answer may be shocking or uncomfortable. sorry, but you didn't have to ask, and you also neither have to read - nor are under an obligation to believe - the answer.

projects require a full-time core members, posted 18 Jan 2005 at 02:07 UTC by lkcl » (Master)

you are no doubt aware that free software projects of this magnitude require, for their continued existance and success, a core of people who work full-time on the project, in order to keep it coherent.

out of all the major codebases, out of _all_ the critical codebases that could make significant inroads into microsoft's monopoly power, DCE has a total of _zero_ full-time developers on it, _zero_ funding, i just don't understand it, i really don't.

anyway.

in 2000, from the mailing list archives at the opengroup's "OpenDCE" section of their site, you can see that an attempt to set up an Open DCE Foundation was initiated.

this effort faltered due to IBM scotching the efforts of the open group to release the code (quite reasonably, because they were still making $billions from it!!!)

last month, we received hints that the code may be available: there was insufficient time to contact the people involved in setting up the ODF again, some of whom we have not heard from for four years.

i would love to see an Open DCE Foundation brought into existence, and i would love to see it funded in order to help it fulfil its charter.

i believe that this code still has value, and to those people who still realise it has value, people like myself - we will do the best - with the resources available and made available - to ensure that its potential is realised.

Difference between OpenOffice / Netscape / Other projects and FreeDCE is: middleware, posted 18 Jan 2005 at 12:12 UTC by lkcl » (Master)

OpenOffice and Netscape were downloaded at their release / announcement and i remember that the openoffice ftp site was crashed within an hour (reason why i did a bittorrent for the slashdot announcement).

netscape removed 128-bit code on release and the race to add it in again and release the first binary was completed within under 12 hours.

these are user-level applications that are obviously important, "do work", "allow people to Do Work", they are in-yer-face, they have an immediate feedback (click, result) that reassures hundreds of thousands of people that they are important to them.

the US Department of Justice even received complaints by companies such as Netscape, AOL, RealPlayer, because their petty little programs were locked out by microsoft's "bundling" strategies. Somewhere, someone in Microsoft is pissing themselves laughing - up until last month.

Because _fortunately_, the European Union case against Microsoft has concluded that the IDL files and specifications are actually very very important, and they _must_ be made available.

it is highly significant that ALL the "biiig players" have dropped out of the EU anti-trust case, leaving the FSFEurope (and one other party i forget which) with the sole responsibility to pursue this. Georg must be absolutely shitting himself at this moment in time.

By contrast and unlike Mozilla and OpenOffice, FreeDCE receives absolutely no funding or attention, because it is "Middleware" and because there are not yet available any "critical" or "in-yer-face" real-world applications that use it, and the reason is quite clear to me.

the entry-level requirements for FreeDCE to take off are considered "too high". FreeDCE is _strategic_ code, it is _developer_-level "developer-assisting" and "enabling" code. and there are some heavy-hitters inside microsoft who appreciated its value and adopted it wholesale.

there is a good reason why DFS is so good: it's because DCE is so comprehensive. DCE allowed Transarc to focus on what they do best: distributed filesystem development.

All other Distributed filesystems have consistently failed to deliver or provide full POSIX filesystem semantics, and the reason is because it takes too long to hand-craft RPC traffic whilst you are still experimenting and debugging the "real" job - the actual service (in the case of transarc, that was DFS - a highly complex job on its own)

without DCE, you have to have employees who are good not only at doing hand-crafting or reinventing RPC mechanisms, but also you have to have employees who are good at the job they were employed to actually _do_.

by contrast, _with_ DCE, if you made a mistake and needed to throw away an entire IDL file which defines an RPC interface, so bloody what? write another one, hit compile, bam, done. tell that to someone who has spent months hand-crafting and reinventing an RPC interface, and they'll look at you like you had two blue heads.

UPDATE, posted 18 Jan 2005 at 12:24 UTC by lkcl » (Master)

the wine team's efforts to reengineer DCOM are at present completely ignoring or dismissing the existance and capability of FreeDCE, even though there exists full, comprehensive and accessible documentation for DCOM from the same site that brought you FreeDCE: The Open Group (see links in reply above)

since i wrote this, i have received confirmation from rob, of codeweavers, that the documentation i found is going to prove absolutely invaluable in the development of Wine's DCOM interoperability. i seek to convince him of the value of using FreeDCE - just as microsoft did _and_ The Open Group have. we'll see! normally i mess these things up by diving in head-first... *sigh*.

http://www.opengroup.org/comsource/datasheet.htm, posted 18 Jan 2005 at 12:34 UTC by lkcl » (Master)

i found _this_ excerpt to be particularly interesting:

"COMsource is based on the code included in Microsofts NT 4.0 version, up to and including service pack 3 (excluding functionality that doesnt map onto UNIXâ; for example, impersonation)."

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page