Generics of a Higher Kind Adriaan Moors Frank Piessens - PDF document

Inthispaper,wefocusonourexperiencewithextend-ingScalawithtypeconstructorpolymorphism,andontheresultinggaininexpressivityofthelanguageasawhole.Asimilarextensioncouldbeaddedto,forexample,Javainthesameway[ 3 ].OurextensionwasincorporatedinScala2.5,whichwasreleasedinMay2007.Themaincontributionsofthispaperareasfollows:  Weillustratetheutilityandpracticalityoftypeconstruc-torpolymorphismusingarealisticexample.  Wedevelopakindsystemthatcapturesbothlowerandupperbounds,andvariancesoftypes.  WesurveyhowtheintegrationwithexistingfeaturesofScala(suchassubtyping,denition-sitevarianceannota-tions,andimplicitarguments)makesthelanguagemorepowerful.  Werelateourexperiencewithimplementingthekindsystemintheopen-sourceScalacompiler.ForthereaderwhoisnotyetfamiliarwithScala,thenextsectionprovidesabriefintroduction.Therestofthispaperisdividedinthreeparts,whicheachconsideradifferentfacetoftheevaluationoftypeconstructorpolymorphism.First,Section 3 demonstratesthatourextensionreducesboiler-platethatarisesfromtheuseofgenericity.Weestablishintu-itionswithasimpleexample,andextendittoarealisticim-plementationofthecomprehensionsfragmentofIterable .Second,wepresentthetypeandkindsystem.Section 4 discussesthesurfacesyntaxinfullScala,andtheunderly-ingmodelofkinds.Basedontheideasestablishedinthetheoreticalpart,Section 5 renesIterable ,sothatitac-commodatescollectionsthatimposeboundsonthetypeoftheirelements.Third,wehavevalidatedthepracticalityofourdesignbyimplementingourextensionintheScalacompiler,andwereportonourexperienceinSection 6 .Throughoutthepaper,wediscussvariousinteractionsoftypeconstructorpolymor-phismwithexistingfeaturesinScala.Section 7 focussesontheintegrationwithScala'simplicits,whichareusedtoen-codeHaskell'stypeclasses.Ourextensionliftsthisencodingtotypeconstructorclasses.Furthermore,duetosubtyping,Scalasupportsabstractingovertypeclasscontexts,sothattheconceptofaboundedmonadcanbeexpressedcleanly,whichisnotpossiblein(mainstreamextensionsof)Haskell.Finally,wesummariserelatedworkinSection 8 andconcludeinSection 9 .2.Prelude:ScalaBasicsThissectionintroducesthebasicsubsetofScala[ 43 , 45 ]thatisusedintheexamplesofthispaper.WeassumefamiliaritywithaJava-likelanguage,andfocusonwhatmakesScaladifferent. 2.1OutlineofthesyntaxAScalaprogramisroughlystructuredasatreeofnesteddenitions.Adenitionstartswithakeyword,followedbyitsname,aclassier,andtheentitytowhichthegivennameisbound,ifitisaconcretedenition.Iftherootofthetreeisthecompilationunit,thenextlevelconsistsofobjects(introducedbythekeywordobject )andclasses(class ,trait ),whichinturncontainmembers.Amembermayagainbeaclassoranobject,aconstantvaluemember(val ),amutablevaluemember(var ),amethod(def ),oratypemember(type ).Notethatatypeannotationalwaysfollowsthename(or,moregenerally,theexpression)thatitclassies.Ontheonehand,Scala'ssyntaxisveryregular,withthekeyword/name/classier/boundentity-sequencebeingitsleadmotif.Anotherimportantaspectofthisregularityisnesting,whichisvirtuallyunconstrained.Ontheotherhand,syntacticsugarenablesexibilityandsuccinctness.Forex-ample,buffer+=10 isshorthandforthemethodcallbuffer.+=(10) ,where+= isauser-denableidentier.2.2FunctionsSinceScalaisafunctionallanguage,functionsarerst-classvalues.Thus,likeaninteger,afunctioncanbewrittendowndirectly:x:Int)x+1 isthesuccessorfunctiononin-tegers.Furthermore,afunctioncanbepassedasanargumenttoa(higher-order)functionormethod.Functionsandmeth-odsaretreatedsimilarlyinScala,themaindifferenceisthatamethodiscalledonatargetobject.Thefollowingdenitionintroducesafunctionlen thattakesaString andyieldsanInt bycallingString 'slength methodonitsarguments : vallen:String)Int=s)s.length Intheclassierofthedenition,thetypeString)Int ,thearrow) isatypeconstructor,whereasitintroducesananonymousfunctionontheright-handside(whereavalueisexpected).Thisanonymousfunctiontakesanarguments oftypeString andreturnss.length .Thus,theapplica-tionlen("four") yields4 .NotethattheScalacompilerinfers[ 46 ]thetypeofthearguments ,basedontheexpectedtypeofthevaluelen .Thedirectionoftypeinferencecanalsobereversed: vallen=(s:String))s.length Theright-handside'sanonymousfunctioncanbeab-breviatedusingsyntacticsugarthatimplicitlyintroducesfunctionalabstraction.ThiscanbethoughtofasturningString 'slength methodintoafunction: vallen:String)Int=_.length Finally,sinceScalaispurelyobject-orientedatitscore,afunctionisrepresentedinternallyasanobjectwithanapply methodthatisderivedstraightforwardlyfromthefunction.Thus,onemoreequivalentdenitionoflen : objectlen{ defapply(s:String):Int=s.length } 2.3Classes,traits,andobjectsInScala,aclasscaninheritfromanotherclassandoneormoretraits.Atraitisaclassthatcanbecomposedwithothertraitsusingmixincomposition.Mixincompositionisarestrictedformofmultipleinheritance,whichavoidsambi-guitiesbylinearisingthegraphthatresultsfromcomposingclassesthatarethemselvescomposites.Thedetailsarenotrelevantforthispaper,andwesimplyrefertobothclassesandtraitsas“classes”.Thefeaturethatisrelevanttothispaper,isthatclassesmaycontaintypemembers.Anabstracttypememberissim-ilartoatypeparameter.Themaindifferencebetweenparam-etersandmembersistheirscopeandvisibility.Atypepa-rameterissyntacticallypartofthetypethatitparameterises,whereasatypemember–likevaluemembers–isencapsu-lated,andmustbeselectedexplicitly.Similarly,typemem-bersareinherited,whiletypeparametersarelocaltotheirclass.ThecomplementarystrengthsoftypeparametersandabstracttypemembersareakeyingredientofScala'srecipeforscalablecomponentabstractions[ 47 ].Typeparametersaremadeconcreteusingtypeapplica-tion.Thus,giventhedenitionclassList[T] ,List isatypeconstructor(ortypefunction),andList[Int] istheapplicationofthisfunctiontotheargumentInt .Abstracttypemembersaremadeconcreteusingabstracttypemem-berrenement,aspecialformofmixincomposition.NotethatList isnowanabstractclass 1 ,sinceithasanabstractmemberT : traitList{ typeT } Thisabstractmemberismadeconcreteasfollows: List{typeT=Int} Notethat,withourextension,typemembersmayalsobeparameterised,asintypeContainer[X] .Methodstypicallydeneoneormorelistsofvaluepa-rameters,inadditiontoalistoftypeparameters.Thus,amethodcanbeseenasavaluethatabstractsovervaluesandtypes.Forexample,defiterate[T](a:T)(next:T)T,done:T)Boolean):List[T] introducesamethodwithonetypeparameterT ,andtwoargumentlists.Methodswithmultipleargumentlistsmaybepartiallyap-plied.Forexample,forsomeobjectx onwhichiterate isdened,x.iterate(0) correspondstoahigher-order 1Forbrevity,weusethetrait keywordinsteadofabstractclass . functionwithtype(Int)Int,Int)Boolean))List[Int] .NotethatthetypeparameterT wasinferredtobeInt fromthetypeofa .Finally,anobject introducesaclasswithasingletoninstance,whichcanbereferredtousingtheobject'sname.3.ReducingCodeDuplicationwithTypeConstructorPolymorphismThissectionillustratesthebenetsofgeneralisinggeneric-itytotypeconstructorpolymorphismusingthewell-knownIterable abstraction.Therstexample,whichisduetoLexSpoon,illustratestheessenceoftheprobleminthesmall.Section 3.1 extendsittomorerealisticproportions.Listing 1 showsaScalaimplementationofthetraitIterable[T] .Itcontainsanabstractmethodfilter andaconveniencemethodremove .Subclassesshouldimplementfilter sothatitcreatesanewcollectionbyretainingonlytheelementsofthecurrentcollectionthatsatisfythepred-icatep .Thispredicateismodelledasafunctionthattakesanelementofthecollection,whichhastypeT ,andreturnsaBoolean .Asremove simplyinvertsthemeaningofthepredicate,itisimplementedintermsoffilter .Naturally,whenlteringalist,oneexpectstoagainre-ceivealist.Thus,List overridesfilter toreneitsresulttypecovariantly.Forbrevity,List 'ssubclasses,whichim-plementthismethod,areomitted.Forconsistency,remove shouldhavethesameresulttype,buttheonlywaytoachievethisisbyoverridingitaswell.Theresultingcodeduplica-tionisaclearindicatorofalimitationofthetypesystem:bothmethodsinList areredundant,butthetypesystemisnotpowerfulenoughtoexpressthemattherequiredlevelofabstractioninIterable .Oursolution,depictedinListing 2 ,istoabstractoverthetypeconstructorthatrepresentsthecontainerofthere-sultoffilter andremove .TheimprovedIterable nowtakestwotypeparameters:therstone,T ,standsforthetypeofitselements,andthesecondone,Container ,repre-sentsthetypeconstructorthatdeterminespartoftheresulttypeofthefilter andremove methods.Morespecically,Container isatypeparameterthatitselftakesonetypepa-rameter.Althoughthenameofthishigher-ordertypeparam-eter(X )isnotneededhere,moresophisticatedexampleswillshowthebenetofexplicitlynaming 2 higher-ordertypepa-rameters.Now,todenotethatapplyingfilter orremove toaList[T] returnsaList[T] ,List simplyinstantiatesIterable 'stypeparametertotheList typeconstructor.Inthissimpleexample,onecouldalsouseaconstructlikeBruce'sMyType [ 9 ].However,thisschemebreaksdowninmorecomplexcases,asdemonstratedinthenextsection. 2InfullScala`_ 'maybeusedasawild-cardnameforhigher-ordertypeparameters. Listing 4 showsaminimalBuildable withanabstractbuild method,andaconveniencemethod,buildWith ,thatcapturesthetypicaluse-caseforbuild .ByanalogytotheprovendesignthatkeepsIterator andIterable separated,Builder andBuildable aremodelledasseparateabstractionsaswell.Inafullimple-mentation,Buildable wouldcontainseveralmoremeth-ods,suchasunfold (thedualoffold [ 22 ]),whichshouldnotclutterthelightweightBuilder interface.NotethatIterable usesatypeconstructormember,Container ,toabstractovertheprecisetypeofthecon-tainer,whereasBuildable usesaparameter.SinceclientsofIterable generallyarenotconcernedwiththeexacttypeofthecontainer(exceptfortheregularitythatisim-posedbyourdesign),itisneatlyencapsulatedasatypemember.Buildable 'sprimarypurposeisexactlytocreateandpopulateaspecickindofcontainer.Thus,thetypeofaninstanceoftheBuildable classshouldspecifythetypeofcontainerthatitbuilds.Thisinformationisstillavailablewithatypemember,butitislessmanifest.Themap /filter /flatMap methodsareimplementedintermsoftheevenmoreexibletriomapTo /filterTo /flatMapTo .Thegeneralisationconsistsofdecouplingtheoriginalcollectionfromtheproducedone–theyneednotbethesame,aslongasthereisawayofbuildingthetargetcol-lection.Thus,thesemethodstakeanextraargumentoftypeBuildable[C] .Section 7 showshowanorthogonalfeatureofScalacanbeusedtorelievecallersfromsupplyingthisargumentexplicitly.Forsimplicity,themapTo methodisimplementedasstraightforwardlyaspossible.ThefilterTo methodshowshowthebuildWith conveniencemethodcanbeused.Theresulttypesofmap ,flatMap ,andtheirgenerali-sationsillustratewhyaMyType -basedsolutionwouldnotwork:whereasthetypeofthis wouldbeC[T] ,theresulttypeofthesemethodsisC[U] :itisthesametypeconstruc-tor,butitisappliedtodifferenttypearguments!Listings 5 and 6 showtheobjectsthatimplementtheBuildable interfaceforList andOption .AnOption correspondstoalistthatcontainseither0or1elements,andiscommonlyusedinScalatoavoidnull 's.Withallthisinplace,List caneasilybeimplementedasasubclassofIterable ,asshowninListing 7 .ThetypeconstructorofthecontainerisxedtobeList itself,andthestandardIterator traitisimplemented.Thisimplementa-tiondoesnotofferanynewinsights,sowehaveomittedit.3.2Example:usingIterableThisexampledemonstrateshowtousemap andflatMap tocomputetheaverageageoftheusersof,say,asocialnet-workingsite.Sinceusersdonothavetoentertheirbirthday,theinputisaList[Option[Date]] .AnOption[Date] traitBuildable[Container[X]]{defbuild[T]:Builder[Container,T]defbuildWith[T](f:Builder[Container,T])Unit):Container[T]={valbuff=build[T]f(buff)buff.finalise()}}traitIterable[T]{typeContainer[X]:Iterable[X]defelements:Iterator[T]defmapTo[U,C[X]](f:T)U)(b:Buildable[C]):C[U]={valbuff=b.build[U]valelems=elementswhile(elems.hasNext){buff+=f(elems.next)}buff.finalise()}deffilterTo[C[X]](p:T)Boolean)(b:Buildable[C]):C[T]={valelems=elementsb.buildWith[T]{buff)while(elems.hasNext){valel=elems.nextif(p(el))buff+=el}}}defflatMapTo[U,C[X]](f:T)Iterable[U])(b:Buildable[C]):C[U]={valbuff=b.build[U]valelems=elementswhile(elems.hasNext){f(elems.next).elements.foreach{el)buff+=el}}buff.finalise()}defmap[U](f:T)U)(b:Buildable[Container]):Container[U]=mapTo[U,Container](f)(b)deffilter(p:T)Boolean)(b:Buildable[Container]):Container[T]=filterTo[Container](p)(b)defflatMap[U](f:T)Container[U])(b:Buildable[Container]):Container[U]=flatMapTo[U,Container](f)(b)} Listing4.Buildable andIterable objectListBuildableextendsBuildable[List]{defbuild[T]:Builder[List,T]=newListBuffer[T]withBuilder[List,T]{//+=isinheritedfromListBuffer(Scalastandardlibrary)deffinalise():List[T]=toList}} Listing5.BuildingaList objectOptionBuildableextendsBuildable[Option]{defbuild[T]:Builder[Option,T]=newBuilder[Option,T]{varres:Option[T]=None()def+=(el:T)=if(res.isEmpty)res=Some(el)elsethrownewUnsupportedOperation-Exception("�1 elements")deffinalise():Option[T]=res}} Listing6.BuildinganOption classList[T]extendsIterable[T]{typeContainer[X]=List[X]defelements:Iterator[T]=newIterator[T]{//standardimplementation}} Listing7.List subclassesIterable eitherholdsadateornothing.Listing 8 showshowtopro-ceed.First,asmallhelperisintroducedthatcomputesthecur-rentageinyearsfromadateofbirth.Tocollecttheknownages,anoptionaldateistransformedintoanoptionalageusingmap .Then,theresultsarecollectedintoalistusingflatMapTo .NotetheuseofthemoregeneralflatMapTo .WithflatMap ,theinnermap wouldhavehadtoconvertitsresultfromanOption toaList ,asflatMap(f) returnsitsresultsinthesamekindofcontainerasproducedbythefunctionf (theinnermap ).Finally,theresultsareaggregatedusingreduceLeft (notshownhere).Thefullcodeoftheexampleisavailableonthepaper'shomepage 3 .NotethattheScalacompilerinfersmostpropertypes(weaddedsomeannotationstoaidunderstanding),butitdoesnot 3 http://www.cs.kuleuven.be/adriaan/?q=genericshk valbdays:List[Option[Date]]=List(Some(newDate("1981/08/07")),None,Some(newDate("1990/04/10")))deftoYrs(bd:Date):Int=//omittedvalages:List[Int]=bdays.flatMapTo[Int,List]{optBd)optBd.map{d)toYrs(d)}(OptionBuildable)}(ListBuildable)valavgAge=ages.reduceLeft[Int](_+_)/ages.length Listing8.Example:usingIterable infertypeconstructorarguments.Thus,typeargumentliststhatcontaintypeconstructors,mustbesuppliedmanually.Finally,theonlytypeconstructorthatarisesintheex-ampleistheList typeargument,astypeconstructorinfer-encehasnotbeenimplementedyet.Thisdemonstratesthatthecomplexityoftypeconstructorpolymorphism,muchlikewithgenericity,isconcentratedintheinternalsofthelibrary.Theupsideisthatlibrarydesignersandimplementershavemorecontrolovertheinterfacesofthelibrary,whileclientsremainblissfullyignorantoftheunderlyingcomplexity.(Asnotedearlier,Section 7 willshowhowtheargumentsoftypeBuildable[C] canbeomitted.)3.3MembersversusparametersTherelativemeritsofabstractmembersandparametershavebeendiscussedindetailbymanyothers[ 8 , 53 , 21 ].TheScalaphilosophyistoembraceboth:sometimesparameter-isationistherighttool,andatothertimes,abstractmembersprovideabettersolution.Technically,ithasbeenshownhowtosafelyencodeparametersasmembers[ 40 ],which–sur-prisingly–wasn'tpossibleinearliercalculi[ 44 ].Ourexampleshaveusedbothstylesofabstraction.Buildable 'smainpurposeistobuildacertaincontainer.Thus,Container isatypeparameter:acharacteristicthatismanifesttoexternalclientsofBuildable ,asitis(syn-tactically)partofthetypeofitsvalues.InIterable atypememberisused,asitsexternalclientsaregenerallyonlyin-terestedinthetypeofitselements.Syntactically,typemem-bersarelessvisible,asIterable[T] isavalidpropertype.Tomakethetypememberexplicit,onemaywriteIterable[T]{typeContainer[X]=List[X]} .Alternatively,theContainer typemembercanbeselectedonasingletontypethatisasubtypeofIterable[T] .4.OfTypesandKindsEventhoughpropertypesandtypeconstructorsareplacedonequalfootingasfarasparametricpolymorphismiscon-cerned,onemustbecarefulnottomixthemup.Clearly,atypeparameterthatstandsforapropertype,mustnotbe traitBuilder[Container[X:B[X]],T:B[T],B[Y]]traitBuildable[Container[X:B[X]],B[Y]]{defbuild[T:B[T]]:Builder[Container,T,B]}traitIterable[T:Bound[T],Bound[X]]{typeContainer[X:Bound[X]]:Iterable[X,Bound]defmap[U:Bound[U]](f:T)U)(b:Buildable[Container,Bound]):Container[U]=...} Listing11.EssentialchangestoextendIterablewithsupportfor(F-)bounds wrong,eventhoughthetypefunctionitselfiswell-kinded,ifitdoessomethingwiththattypeconstructorthatwouldbeadmissiblewithatypeofkind*!* ,butnotwithatypeofkind*(Number)!* ,suchasapplyingittoString .Iftherst,erroneous,typeapplicationwereconsideredwell-kinded,typeapplicationwouldnotbekind-preserving,asitwouldturnawell-kindedtypeintoanonsensical,ill-kinded,one(suchasNumericList[String] ).Asourkindsystemiscloselyrelatedtodependentlytypedlambdacalculuswithsubtyping,itisreasonabletoas-sumethatitissound.Provingthisconjecture,aswellasthemorefamiliarmeta-theoreticresults,isongoingwork.Theunderlyingtheory–anobject-orientedcalculus–hasbeendescribedinearlierwork[ 40 ].Finally,itisimportanttonotethatkindunsoundnessresultsintypeapplications“goingwrong”atcompiletime.Thus,theproblemislessseverethanwithtypeunsoundness,buttheseerrorscanbedetectedearlierinthedevelopmentprocess,withouteffortfromtheprogrammer.5.BoundedIterableAsmotivatedinSection 4.4 ,inorderforIterable tomodelcollectionsthatimposean(F-)boundonthetypeoftheirelements,itmustaccommodatethisboundfromthestart.ToallowsubclassesofIterable todeclarean(F-)boundonthetypeoftheirelements,Iterable mustabstractoverthisbound.Listing 11 generalisestheinterfaceoftheorig-inalIterable fromListing 4 .Theimplementationisnotaffectedbythischange.Listing 12 illustratesvariouskindsofsubclasses,includ-ingList ,whichdoesnotimposeaboundonthetypeofitselements,andthususesAny asitsbound(Any andNothing arekind-overloaded).NotethatNumericList canalsobederived,byencodingtheanonymoustypefunctionX!Number asWrap1[Number]#Apply .Again,theclientofthecollectionsAPIisnotexposedtotherelativecomplexityofListing 11 .However,without classList[T]extendsIterable[T,Any]{typeContainer[X]=List[X]}traitOrderedCollection[T:Ordered[T]]extendsIterable[T,Ordered]{typeContainer[X:Ordered[X]]:OrderedCollection[X]}traitWrap1[T]{typeApply[X]=T}traitNumberclassNumericList[T:Number]extendsIterable[T,Wrap1[Number]#Apply]{typeContainer[X:Number]=NumericList[X]} Listing12.(Bounded)subclassesofIterable it,asignicantfractionofthecollectionclassescouldnotbeuniedunderthesameIterable abstraction.Thus,theclientsofthelibrarybenet,asauniedinterfaceforcol-lections,whethertheyconstrainthetypeoftheirelementsornot,meansthattheyneedtolearnfewerconcepts.Alternatively,itwouldbeinterestingtointroducekind-levelabstractiontosolvethisproblem.Tentatively,Iter-able andList couldthenbeexpressedas:traitIterable[T:ElemK,ElemK:Kind]classList[T]extendsIterable[T,*] Thisapproachismoreexpressivethansimplyabstractingovertheupperboundontheelementtype,astheintervalkindcanexpresslowerandupperboundssimultaneously.Thiswouldbecomeevenmoreappealinginalanguagethatallowstheusertodenenewkinds[ 51 ].6.FullScalaInthissectionwediscussourexperiencewithextendingthefullScalacompilerwithtypeconstructorpolymorphism.Asdiscussedbelow,theimpact 5 ofourextensionismostlyre-strictedtothetypechecker.Finally,welistthelimitationsofourimplementation,anddiscusstheinteractionwithvari-ance.Theimplementationsupportsvarianceannotationsonhigher-ordertypeparameters,butthishasnotbeeninte-gratedintheformalisationyet.6.1ImplementationExtendingtheScalacompilerwithsupportfortypeconstruc-torpolymorphismcamedowntointroducinganotherlevelofindirectioninthewell-formednesschecksfortypes.Onceabstracttypescouldbeparameterised(asimpleextensiontotheparserandtheabstractsyntaxtrees),the 5Theinitialpatchtothecompilercanbeviewedat http://lampsvn.epfl.ch/trac/scala/changeset/10642 defmax[T](x:T,y:T)(implicitconv:T)Ord[T]):T=if(x=y)yelsex Listing19.Desugaringviewbounds defmax[T](x:T,y:T)(c:T)Ord[T]):T=if(c(x).=(y))yelsex Listing20.Makingimplicitsexplicit tosupplytheimplicitconversionexplicitly:max(dateA,dateB)(dateAsOrd) .7.2.3ConditionalimplicitsBydeningimplicitmethodsthatthemselvestakeimplicitparameters,Haskell'sconditionalinstancedeclarationscanbeencoded:instanceOrda)Ord(Lista)where(=)=... ThisisencodedinScalaas:implicitdeflistAsOrd[T](self:List[T])(implicitv:T)Ord[T])=newOrd[List[T]]{def=(other:List[T])=//compareelementsinselfandother} Thus,twolistswithelementsoftypeT canbecomparedaslongastheirelementsarecomparable.Typeclassesandimplicitsbothprovidead-hocpolymor-phism.Likeparametricpolymorphism,thisallowsmethodsorclassestobeapplicabletoarbitrarytypes.However,para-metricpolymorphismimpliesthatamethodoraclassistrulyindifferenttotheactualargumentofitstypeparam-eter,whereasad-hocpolymorphismmaintainsthisillusionbyselectingdifferentmethodsorclassesfordifferentactualtypearguments.Thisad-hocnatureoftypeclassesandimplicitscanbeseenasaretroactiveextensionmechanism.InOOP,vir-tualclasses[ 48 , 20 ]havebeenproposedasanalternativethatisbettersuitedforretroactiveextension.However,ad-hocpolymorphismalsoallowstypestodrivetheselectionoffunctionalityasdemonstratedbytheselectionof(im-plicit)instancesofBuildable[C] inourIterable exam-ple 6 .Buildable clearlycouldnotbetrulypolymorphicinitsparameter,asthatwouldimplythattherecouldbeoneBuildable thatknewhowtosupplyastrategyforbuildinganytypeofcontainer. classMonadmwhere(��=)::ma!(a!mb)!mbdata(Orda))Seta=...instanceMonadSetwhere--(��=)::Seta!(a!Setb)!Setb Listing21.Set cannotbemadeintoaMonad inHaskell traitMonad[A,M[X]]{def��=[B](f:A)M[B]):M[B]} Listing22.Monad inScala traitBoundedMonad[A:Bound[A],M[X:Bound[X]],Bound[X]]{def&#x]TJ/;དྷ ;.96;d T; 6.;≣&#x 0 T; [00;&#x]TJ/;དྷ ;.96;d T; 6.;≣&#x 0 T; [00;=[B:Bound[B]](f:A)M[B]):M[B]}traitSet[T:Ord[T]]implicitdefSetIsBoundedMonad[T:Ord[T]](s:Set[T]):BoundedMonad[T,Set,Ord]=... Listing23.Set asaBoundedMonad inScala 7.3ExceedingtypeclassesAsshowninListing 21 ,Haskell'sMonad abstraction[ 55 ]doesnotapplytotypeconstructorswithaconstrainedtypeparameter,suchasSet ,asexplainedbelow.ResolvingthisissueinHaskellisanactiveresearchtopic[ 15 , 16 , 29 ].Inthisexample,theMonad abstraction 7 doesnotaccom-modateconstraintsonthetypeparameterofthem typecon-structorthatitabstractsover.SinceSet isatypeconstructorthatconstrainsitstypeparameter,itisnotavalidargumentforMonad 'sm typeparameter:ma isallowedforanytypea ,whereasSeta isonlyallowedifa isaninstanceoftheOrd typeclass.Thus,passingSet asm couldleadtoviolatingthisconstraint.Forreference,Listing 22 showsadirectencodingoftheMonad typeclass.TosolvetheprobleminScala,wegeneraliseMonad toBoundedMonad inListing 23 todealwithboundedtypeconstructors.Finally,theencodingfromSection 7.2 isusedtoturnaSet intoaBoundedMonad . 6Java'sstaticoverloadingmechanismisanotherexampleofad-hocpoly-morphism.7Infact,themaindifferencebetweenourIterable andHaskell'sMonad isspelling. 8.RelatedWork8.1RootsofourkindsSincetheseminalworkofGirardandReynoldsintheearly1970's,fragmentsofthehigher-orderpolymorphiclambdacalculusorSystemF![ 23 , 50 , 7 ]haveservedasthebasisformanyprogramminglanguages.ThemostnotableexampleisHaskell[ 28 ],whichhassupportedhigher-kindedtypesforover15years[ 27 ].AlthoughHaskellhashigher-kindedtypes,iteschewssubtyping.Mostoftheuse-casesforsubtypingaresubsumedbytypeclasses,whichhandleoverloadingsystematically[ 56 ].However,itisnot(yet)possibletoabstractoverclasscontexts[ 29 , 32 , 34 , 15 ].Inoursetting,thiscorrespondstoabstractingoveratypethatisusedasabound,asdiscussedinSection 7.3 .Theinteractionbetweenhigher-kindedtypesandsubtyp-ingisawell-studiedsubject[ 13 , 12 , 10 , 49 , 17 ].Asfarasweknow,noneoftheseapproachescombineboundedtypeconstructors,subkinding,subtypingandvariance,althoughallofthesefeaturesareincludedinatleastoneofthem.AsimilarityofinterestisCardelli'snotionofpowertypes[ 11 ],whichcorrespondstoourbounds-trackingkind*(L,U) .Insummary,thepresentedtypesystemcanbethoughtofastheintegrationofanobject-orientedsystemwithPolar-izedF!sub[ 52 ],Cardelli'spowertype,andsubkinding.Sub-kindingisbasedonintervalinclusionandthetranspositionofsubtypingofdependentfunctiontypes[ 4 ]tothelevelofkinds.8.2TypeconstructorpolymorphisminOOPL'sLanguageswithvirtualtypesorvirtualclasses,suchasgbeta[ 20 ],canencodetypeconstructorpolymorphismthroughab-stracttypemembers.Theideaistomodelatypeconstruc-torsuchasList asasimpleabstracttypethathasatypememberdescribingtheelementtype.SinceScalahasvirtualtypes,List couldalsobedenedasaclasswithanabstracttypememberinsteadofasatype-parameterisedclass:abstractclassList{typeElem} Then,aconcreteinstantiationofList couldbemodelledasatyperenement,asinList{typeElem=String} .ThecrucialpointisthatinthisencodingList isatype,notatypeconstructor.Sorst-orderpolymorphismsufcestopasstheList constructorasatypeargumentoranabstracttypememberrenement.Comparedtotypeconstructorpolymorphism,thisencod-inghasaseriousdisadvantage,asitpermitsthedenitionofcertainaccidentallyemptytypeabstractionsthatcannotbeinstantiatedtoconcretevalueslateron.Bycontrast,typeconstructorpolymorphismhasakindsoundnesspropertythatguaranteesthatwell-kindedtypeapplicationsneverre-sultinnonsensicaltypes.Typeconstructorpolymorphismhasrecentlystartedtotrickledowntoobject-orientedlanguages.CremetandAl- therr'sworkonextendingFeatherweightGenericJavawithhigher-kindedtypes[ 18 ]partlyinspiredthedesignofoursyntax.However,sincetheyextendJava,theydonotmodeltypemembersandpath-dependenttypes,denition-sitevari-ance,orintersectiontypes.Theydoprovidedirectsupportforanonymoustypeconstructors.Furthermore,althoughtheirworkdemonstratesthattypeconstructorpolymorphismcanbeintegratedintoJava,theyonlyprovideaprototypeofacompilerandaninterpreter.However,theyhavedevelopedamechanisedsoundnessproofandapencil-and-paperproofofdecidability.Finally,webrieymentionOCamlandC++.C++'stem-platemechanismisrelated,but,whiletemplatesareveryexible,thiscomesatasteepprice:theycanonlybetype-checkedaftertheyhavebeenexpanded.Recentworkon“concepts”alleviatesthis[ 25 ].InOCaml(asinML),typeconstructorsarerst-order.Thus,althoughatypeof,e.g.,kind*!*!* issup-ported,typesofkind(*!*)!*!* cannotbeex-presseddirectly.However,MLdialectsthatsupportapplica-tivefunctors,suchasOCamlandMoscowML,canencodetypeconstructorpolymorphisminmuchthesamewayaslanguageswithvirtualtypes.9.ConclusionGenericityisaproventechniquetoreducecodeduplicationinobject-orientedlibraries,aswellasmakingthemeasiertousebyclients.Theprimeexampleisacollectionslibrary,whereclientsnolongerneedtocasttheelementstheyre-trievefromagenericcollection.Unfortunately,thoughgenericityisextremelyuseful,therst-ordervariantisself-defeatinginthesensethatabstract-ingoverpropertypesgivesrisetotypeconstructors,whichcannotbeabstractedover.Thus,byusinggenericitytore-ducecodeduplication,otherkindsofboilerplatearise.Typeconstructorpolymorphismallowstofurthereliminatetheseredundancies,asitgeneralisesgenericitytotypeconstruc-tors.Aswithgenericity,mostusecasesfortypeconstructorpolymorphismariseinlibrarydesignandimplementation,whereitprovidesmorecontrolovertheinterfacesthatareexposedtoclients,whilereducingcodeduplication.More-over,clientsarenotexposedtothecomplexitythatisin-herenttotheseadvancedabstractionmechanisms.Infact,clientsbenetfromthemorepreciseinterfacesthatcanbeexpressedwithtypeconstructorpolymorphism,justlikegenericityreducedthenumberofcaststhatclientsofacol-lectionslibraryhadtowrite.WeimplementedtypeconstructorpolymorphisminScala2.5.TheessenceofoursolutioncarriesovereasilytoJava,seeAltherretal.foraproposal[ 3 ].Finally,wehaveonlyreportedononeofseveralappli-cationsthatwehaveexperimentedwith.Embeddeddomainspeciclanguages(DSL's)[ 14 ]areanotherpromisingappli- cationareaoftypeconstructorpolymorphism.Wearecur-rentlyapplyingtheseideastoourparsercombinatorlibrary,aDSLforwritingEBNFgrammarsinScala[ 39 ].Hofer,Ostermannetal.areinvestigatingsimilarapplications[ 26 ],whichcriticallyrelyontypeconstructorpolymorphism.AcknowledgmentsTheauthorswouldliketothankDaveClarke,MarkovanDooren,BurakEmir,ErikErnst,BartJacobs,AndreasRoss-berg,JanSmans,andLexSpoonfortheirinsightfulcom-mentsandinterestingdiscussions.Wealsogratefullyac-knowledgetheScalacommunityforprovidingafertiletestbedforthisresearch.Finally,wethankthereviewersfortheirdetailedcommentsthathelpedusimprovethepaper.AnolderversionofthispaperwaspresentedattheMPOOLworkshop[ 38 ].TherstauthorissupportedbyagrantfromtheFlemishIWT.Partofthereportedworkwasperformedduringa3-monthstayatEPFL.References [1] M.AbadiandL.Cardelli.Atheoryofprimitiveobjects:Second-ordersystems.Sci.Comput.Program.,25(2-3):81–116,1995. [2] M.AbadiandL.Cardelli.Atheoryofprimitiveobjects:Untypedandrst-ordersystems.Inf.Comput.,125(2):78–102,1996. [3] P.AltherrandV.Cremet.Addingtypeconstructorparameter-izationtoJava.AcceptedtotheworkshoponFormalTech-niquesforJava-likePrograms(FTfJP'07)attheEuropeanConferenceonObject-OrientedProgramming(ECOOP),2007. [4] D.AspinallandA.B.Compagnoni.Subtypingdependenttypes.Theor.Comput.Sci.,266(1-2):273–309,2001. [5] G.M.Bierman,E.Meijer,andW.Schulte.TheessenceofdataaccessinComega.InA.P.Black,editor,ECOOP,volume3586ofLectureNotesinComputerScience,pages287–311.Springer,2005. [6] G.Bracha.ExecutablegrammarsinNewspeak.Electron.NotesTheor.Comput.Sci.,193:3–18,2007. [7] K.B.Bruce,A.R.Meyer,andJ.C.Mitchell.Thesemanticsofsecond-orderlambdacalculus.Inf.Comput.,85(1):76–134,1990. [8] K.B.Bruce,M.Odersky,andP.Wadler.Astaticallysafealternativetovirtualtypes.InE.Jul,editor,ECOOP,volume1445ofLectureNotesinComputerScience,pages523–549.Springer,1998. [9] K.B.Bruce,A.Schuett,andR.vanGent.PolyTOIL:Atype-safepolymorphicobject-orientedlanguage.InW.G.Olthoff,editor,ECOOP,volume952ofLectureNotesinComputerScience,pages27–51.Springer,1995. [10] P.S.Canning,W.R.Cook,W.L.Hill,W.G.Olthoff,andJ.C.Mitchell.F-boundedpolymorphismforobject-orientedprogramming.InFPCA,pages273–280,1989. [11] L.Cardelli.Structuralsubtypingandthenotionofpowertype.InPOPL,pages70–79,1988. [12] L.Cardelli.Typesfordata-orientedlanguages.InJ.W.Schmidt,S.Ceri,andM.Missikoff,editors,EDBT,volume303ofLectureNotesinComputerScience,pages1–15.Springer,1988. [13] L.CardelliandP.Wegner.Onunderstandingtypes,dataabstraction,andpolymorphism.ACMComputingSurveys,17(4):471–522,1985. [14] J.Carette,O.Kiselyov,andC.chiehShan.Finallytagless,partiallyevaluated.InZ.Shao,editor,APLAS,volume4807ofLectureNotesinComputerScience,pages222–238.Springer,2007. [15] M.Chakravarty,S.L.P.Jones,M.Sulzmann,andT.Schri-jvers.Classfamilies,2007.OntheGHCDeveloperwiki, http://hackage.haskell.org/trac/ghc/wiki/TypeFunctions/ClassFamilies . [16] M.M.T.Chakravarty,G.Keller,S.L.P.Jones,andS.Marlow.Associatedtypeswithclass.InJ.PalsbergandM.Abadi,editors,POPL,pages1–13.ACM,2005. [17] A.B.CompagnoniandH.Goguen.Typedoperationalseman-ticsforhigher-ordersubtyping.Inf.Comput.,184(2):242–297,2003. [18] V.CremetandP.Altherr.Addingtypeconstructorparame-terizationtoJava.JournalofObjectTechnology,7(5):25–65,June2008.SpecialIssue:WorkshoponFTfJP,ECOOP07.http://www.jot.fm/issues/issue 2008 06/article2/. [19] B.Emir,A.Kennedy,C.V.Russo,andD.Yu.VarianceandgeneralizedconstraintsforC#generics.InD.Thomas,editor,ECOOP,volume4067ofLectureNotesinComputerScience,pages279–303.Springer,2006. [20] E.Ernst.gbeta–aLanguagewithVirtualAttributes,BlockStructure,andPropagating,DynamicInheritance.PhDthesis,DepartmentofComputerScience,UniversityofAarhus, Arhus,Denmark,1999. [21] E.Ernst.Familypolymorphism.InJ.L.Knudsen,editor,ECOOP,volume2072ofLectureNotesinComputerScience,pages303–326.Springer,2001. [22] J.GibbonsandG.Jones.Theunder-appreciatedunfold.InICFP,pages273–279,1998. [23] J.Girard.Interpretationfonctionelleeteliminationdescoupuresdel'arithmetiqued'ordresuperieur.Thesed'´Etat,ParisVII,1972. [24] J.Gosling,B.Joy,G.Steele,andG.Bracha.Java(TM)LanguageSpecication,The(3rdEdition)(Java(Addison-Wesley)).Addison-WesleyProfessional,2005. [25] D.Gregor,J.J¨arvi,J.G.Siek,B.Stroustrup,G.D.Reis,andA.Lumsdaine.Concepts:linguisticsupportforgenericprogramminginC++.InP.L.TarrandW.R.Cook,editors,OOPSLA,pages291–310.ACM,2006. [26] C.Hofer,K.Ostermann,T.Rendel,andA.Moors.Polymor-phicembeddingofDSLs.InY.SmaragdakisandJ.Siek,editors,GPCE.ACM,2008.Toappear.