Sparse Vector Storage: Setting Compatible Parameter on OCI Autonomous Database

begin
    execute immediate q'~
        create table my_sparse_vectors(
            vec vector(4, int8, sparse)
        )
        ~';
end;
/
ORA-00406: COMPATIBLE parameter needs to be 23.6.0.0.0 or greater
ORA-00722: Feature "SQL VECTOR SPARSE type"

You can’t always get what you want…. Sometimes new features that come with a release update require the compatible initialization parameter to be set to a specific release or greater.

VirtualBox 23ai

Setting this parameter from a privileged account in a pluggable database won’t work:

ALTER SYSTEM SET COMPATIBLE = '23.7.0.0' SCOPE=SPFILE;

Error report -
ORA-65040: Operation is not allowed from within a pluggable database.

https://docs.oracle.com/error-help/db/ora-65040/65040. 00000 -  "Operation is not allowed from within a pluggable database."
*Cause:    An operation was attempted that can only be performed in the root
           or application root container.
*Action:   Switch to the root or application root container to perform the
           operation.

If you are running a 23AI Free database on virtual box, you will have to look at the /opt/Oracle/product/23ai/dbhomefree/network/admin/tnsnames.ora file to discover that the CDB service name is FREE. The PDB service name is FREEPDB1 as noted in the readme file that comes on the home desktop in the virtual appliance.

Reconnect as the sys user to the CDB and the alter system command works:

ALTER SYSTEM SET COMPATIBLE = '23.7.0.0' SCOPE=SPFILE;
System altered.

The 23.7 release of virtual box development environment actually comes with the compatible parameter set to 23.6.0.0, so its not necessary to do anything to work with sparse vector storage in this environment, but its good to know how. The 23.6 release on VirtualBox also had this parameter already set.

OCI Autonomous Database

Since the compatible parameter is already set in Virtualbox releases of 23ai, you won’t be able to reproduce the compatible error there. If you are doing your research in the OCI Autonomous Database with an Always Free instance, the story is a bit longer.

When the feature for sparse vector storage was first announced in the release notes documentation, the OCI database would not allow creating a table with sparse vectors with an odd error:

Error report -
ORA-45493: operation was unsupported during rolling upgrade
ORA-00722: Feature "SQL VECTOR SPARSE format"
45493. 00000 -  "operation was unsupported during rolling upgrade"
*Document: NO
*Cause:    An unsupported operation was attempted during logical standby
           rolling upgrade.
*Action:   Wait until the rolling upgrade has completed or clear the
           BLOCK_UNSUPPORTED rolling upgrade parameter and retry
           the rolling upgrade. Then retry the operation.

Since having a free OCI account with a couple of Always Free database instances also gives me access to My Oracle Support and the ability to file Service Requests, I reported this bug on the OCI platform and the development team had resolved it around the time the 23.7 virtual box was released.

Trying to test if the ‘rolling upgrade’ issue was resolved led to the new compatible parameter issue that I introduced this post with.

The interesting thing about the Autonomous Database is that there is no access to the CDB at all. I tried to alter the system with an account that I had given PDB DBA privileges, but that doesn’t work:

ORA-01031: insufficient privileges

Logging in with the primary ADMIN account also failed due to insufficient privileges.

By this time I received a reply on my Service Request about the new insufficient privileges error and the development team confirmed that there would be no way to set the compatible parameter in the Autonomous Database and they are currently working on this issue.

I will update this post when I get a solution from support for creating tables with sparse vector storage. The solution should be interesting, either a sufficiently privileged account should be able to set the parameter or the automatic release updates for autonomous database will set the compatible parameter as well.

Summary

If you try sometimes, you get what you need… Back in my local VirtualBox environment all is well and I am happily researching the Sparse vector storage feature:

begin
    execute immediate q'~
        create table my_sparse_vectors(
            vec vector(4, int8, sparse)
        )
        ~';
end;
/

Table MY_SPARSE_VECTORS created.

In addition to understanding how to set the compatible parameter in an on-premises database instance, the main takeaway from this post should be an understanding of how much Oracle has empowered the individual developer at this point in time.

Unlike previous versions, there have been regular releases of the new release updates on the 23ai Free VirtualBox appliances.

select banner_full from v$version;
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.7.0.25.01

With an Always Free autonomous database, its free to access the Enterprise Edition release on the OCI cloud and automatically get release updates and patches.

select banner_full from v$version;
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems
Version 23.7.0.25.03"

The liveSQL site has also been upgraded to Oracle 23ai, exposing the new Extreme Performance Enterprise Edition with some limitations due to the implementation of liveSQL. This is a great site allowing access to a session running on the Oracle database with a simple website that is worth a separate post exploring the capabilities exposed here.

select banner_full from v$version;
Oracle Database 23ai EE Extreme Perf Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems
Version 23.5.0.24.07

With a free OCI cloud account developers have the unprecedented privilege of creating Service Requests with Oracle support. Previously, this could only be done for customers who purchased Oracle database licenses.

Oracle continues to make their database technology freely available to developers who want to research the capabilities and features of the database and Pl/sql themselves before introducing a technique in their workplace environment.

As Oracle developers, we are the first to explore new features on the cutting edge. Because of this, we can run into bugs first, before they are discovered in production environments. Allowing individual developers access to filing Service Requests is the best part of a free OCI account, and the best way for Oracle to continue leading the way in database technology.

Enjoy your explorations into the groundbreaking new features of the Oracle database, and create an OCI account for yourself so that you can help pave the way for everyone to use these new features. As a global community of developers, we can lend our combined strength to helping Oracle database continue to be the database of choice for any enterprise.

–Anthony Harper

Note: Update as of April 17, 2025

It looks like there is a new update for OCI Autonomous database, we are now on 23.8!

select banner_full from v$version;

Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems Version 23.8.0.25.05

Looking at the compatible parameter, it has been updated to 23.5:

SELECT value 
FROM v$parameter 
WHERE name = 'compatible';
                                                                    
23.5.0  

Creation of tables with sparse vector storage now works without issue on OCI:

create table if not exists quiz_vectors(
    v vector(4,int8, sparse)
)
/

Table QUIZ_VECTORS created.

Great, now OCI Autonomous database supports tables with sparse vector storage! This already works on the 23.7 VirtualBox. Time to work on researching sparse vector storage!

–Anthony Harper


Source code for this article can be found at https://github.com/gaiansentience/practicalplsql

Discussion and Comments

4 responses to “Sparse Vector Storage: Setting Compatible Parameter on OCI Autonomous Database”

  1. iudithd5bf8e4d8d Avatar
    iudithd5bf8e4d8d

    Hello Anthony,

    I really like your optimistic and constructive approach to all the new developments 🙂

    Unfortunately, not everyone is so lucky as to have access to all these new environments, due to various technical reasons that are worth a separate discussion for each …

    Regarding the autonomous databases that you are testing, I think that all these compatibility issues should be managed by Oracle, as they emphasize in each webinar on the ADB topics, so that you, the developer, have the environment completely ready to go for each release, with all the parameters properly set, a.s.o.
    I don’t understand why this was not the case for your environment …

    Regarding specifically LiveSQL, that you mentioned, I have just two short comments:

    1. The “classic” web site is still in in version 19c (19.17) and it will probably NOT be upgraded any time soon,
    or maybe not at all.
    This is what I was told several weeks ago, when I asked for some enhancement, as I have done several times along the many years that I use this environment very satisfactorily.

    2. Instead, they told me that the V2 environment, which they strongly encourage developers to use for testing, will be soon open for new enhancement suggestions, a.s.o.
    This environment works differently from the old one from many points of view, so I only used it very sparsely.

    Yesterday however, I discovered that it is not possible at all to login to the V2 environment, plain and simply NOT !

    The two environments seemed to be “somehow connected” to each other, in the sense that even when the V2 environment still worked, it was not possible to simultaneously be logged in to both, which I don’t understand why,
    as they are different databases, with different versions …
    I sent a Bug Feedback to the LiveSQL team to report the login problem and also opened a request on another forum to where I was directed from the V2 web page, let’s see what happens.

    In the meantime I found some other “toy” online environments that do work, with 23c or 23ai (not the last release, though), but they do have some privilege restrictions, I guess many of them because those who maintain these environments have not given too much thought to these last versions enhancements.

    I also know of some other solutions, that I have not explored (yet), because I was pretty sure that the LiveSQL
    environment will be soon upgraded and maybe even enhanced to our satisfaction.
    Also, this is the environment where sample cases are supposed to be created for questions submitted to the Asktom team … so this is also currently limited by the version problem …

    In summary, in the meantime, with all the bells and whistles, I can only use other (not Oracle managed) environments,
    plain and simple !

    Sorry for troubling you with all my complaints … but I really feel frustrated and have the feeling that nobody is listening there to “outsiders” like me … LiveSQL is probably only a “niche activity” today … not one into which too much diligence is to be invested …

    Just as you say in this post and in the same spirit, my purpose was always to try to be useful and this is why, in spite of my limited access, I always tried to submit enhancement requests that I thought would be useful for other developers too …

    Thank you for all your great work and interesting posts, that I am always following with much interest 🙂

    Cheers & Best Regards,
    Iudith

    Like

    1. Anthony Harper Avatar

      Iudith,
      I appreciate your frustration at not feeling able to easily obtain the current Oracle release for your research.

      Your insights on Oracle features are always insightful and challenging, it does seem like a disservice to our community that you feel hampered by the lack of a suitable research environment.

      I checked liveSQL this afternoon. The default is to land the user on the new 23ai site, and the login is working. The interface is completely redesigned, but I think you will find it to be greatly improved compared to the old interface. The feeling is much closer to SQL developer now, and very parallel to the look and feel of the Visual Studio Code add-in.

      I agree that logging in again when switching to the ‘classic’ 19c database is awkward and it would be easier to use if the transition between instances were more seamless. Personal script libraries developed in the classic liveSQL site are also not available in the new site. It looks like we will have to download scripts and then reupload to the new site to be able to use them, Chris Saxon announced recently that he is working on migrating the DevGym workout scripts to the new site.

      The limitations of creating a web site where anyone can log into a database and execute SQL or PL/SQL are more difficult for me to work with, as I like the ability to login with administrative privileges on my development environment to adjust things. Testing inter-schema grants for users and roles is also not possible in this invironment.

      As I have stated, I love the autonomous environment for Always Free databases on OCI primarily because they patch the database for me and apply release updates. I knew that I could not change the compatibility settings on the Autonomous Database, so I was surprised to encounter a compatibility parameter issue there. My intuition is that they will apply the most recent compatibility setting with release updates, and we won’t be able to maintain older compatibiity settings on autonomous instances. Since you get two Always Free databases, I also have a 19c instance. That instance is up to date with the latest release updates, and Chris found that the backport of scalar macros that happened in one of the recent release updates was only in the Autonomous database (He is running 19c EE Extreme Edition and scalar macros were not supported as of release update 25.

      I have been using the OCI free instance for 19c for quite a while, as I dont really have any ambitions to be a dba and apply the release updates myself. This was the strategic advantage of maintaining the OCI free account over VirtualBox, which is my preferred environment (because I can take a copy of the appliance before doing anything drastically experimental). The 19c Developer Days appliance was never updated, making the only ways to explore later 19c features liveSQL and OCI Always Free instances. Things are completely different with the 23ai VirtualBox appliance. Oracle has been updating the distribution for this appliance with every 23ai release update, and it is just as current as the OCI options.

      As long as 23ai Enterprise Edition On Premises is not released for general download, the only free options for me will continue to be OCI Always Free instances, VirtualBox Appliances and LiveSQL. I believe that the containerized distribution is also being kept up to date, but I don’t want to install anything more than VirtualBox on my laptop for the sake of simplicity. And when the EE is available for download, it would have to be patched with release updates. As a developer, I don’t have the time to install and maintain the database myself. It was really not very fun to upgrading Python and install OML4PY to generate the full set of import compatible ML models to explore vector search inside the database, which is why I share all of the sample ML models that I could generate with all developers.

      There are no costs associated with the VirtualBox appliances or the OCI Always Free databases. I really do hope that you can overcome any other factors that are preventing you from using these environments because our community definitely benefits when you are able to apply your considerable skills to any problem.

      Best Regards,
      Anthony

      Like

  2. iudithd5bf8e4d8d Avatar
    iudithd5bf8e4d8d

    Hi Anthony,

    Thanks a lot for your so detailed feedback :):)

    Yes, by now they have solved the login problem of LiveSQL :):)
    There is no automatic landing, on the main page you can choose one or the other.
    The [sign-in] button is available both on the main page and on the V2 page.

    And, yes … they tried to make the V2 environment more close to one or more of the client-server tools,
    which of course cannot satisfy everybody, surely not me, who was always used to the simplicity of the least sophisticated tools and using my own scripts (and typing) for everything.
    I am “old-school”, so I find too much “environmental noise”, like too much display which is not on-demand,
    and things just happening under your fingers rather disturbing …

    I learned to live with the limitations of LiveSQL … some of them indeed quite annoying …
    For example, the problem of using GRANT-s was added at a certain point … I still remember the early times
    when you could open two sessions (ex. using two different browsers) and were able to execute GRANT-s from one to the other.
    I raised this problem and Chris even told me that “I cannot imagine how some bad guys might exploit GRANT-s
    for creating serious security issues …” … and I admit, I cannot !
    As long as my entire schema is volatile and I can only make grants on my own objects, which are anyway limited on what they can do, I don’t see any issue here …

    I liked for example the complete volatility of the LiveSQL environment … so that I could perform full tests without leaving anything behind, to have to be cleaned up … maybe a feature that could be implemented as a choice in the new environment as well.

    I am sure that many of today’s limitations of LiveSQL can be lifted, of course, at the cost of having one or more dedicated developers assigned to checking and implementing all such requests.
    ( if Oracle was “our store”, and I was some years younger, I think I would have liked to be one of those … ).
    As a quick example, there exist many DBMS packages that contain specific subprograms requiring more elevated privileges, and, therefore, it was decided NOT to grant execute on them at all.
    I don’t think that this should be an issue, because the regular users do not have those privileges anyway,
    but, even if this does not hold because of the LiveSQL internal implementation itself, wrapper packages could be created which do not contain the problematic subprograms.
    And, there are many more …

    I also hope that I will maybe soon be able to use the autonomous environment.
    I also like the idea of somebody managing the database for me …
    This is why a long time ago I was almost on the point of starting to use VirtualBox … but also stopped somewhere in the middle … this was still before I retired from work …
    I only performed then a quick test using 12c, and it was extremely slow … even on a strong enough desktop machine … it required an enormous memory, much more than documented, for running smoothly.
    11g was a little bit faster … But, anyway, I still consider that you should know a little bit of Linux “to manage yourself”, and not just dumbly using an appliance …
    Therefore, when LiveSQL came in shortly afterwards, I felt it like a fresh breeze :):)

    Yes … I myself like to have admin privileges … it suffices to tell you that, when still at work, I had even implemented
    some application upgrade routines by performing carefully crafted updates to some SYS tables … to solve some problems for which the regular API-s were not sufficient.

    Regarding the COMPATIBILITY parameter:
    I don’t exactly understand why is it required to be set at CDB level, and not at PDB level ?
    If it can be controlled using ALTER SYSTEM, this means that it is just a matter of behavior, to control using features
    that are anyway available in the binaries.
    Like other features, I guess that this might change in a further release.

    Regarding the ML models …
    I think Oracle could “do the work for us” to prepare all these load-compatible models and make them available
    to all the users, or maybe even load them into the database, without everyone having to become a Python specialist to handle the issue.
    It is instructive to do it, of course, but definitely not everyone’s “piece of cake”.

    Wish you a lot of fun and always useful research and let’s hope for the best 🙂

    Cheers & Best Regards,
    Iudith

    Like

    1. Anthony Harper Avatar

      Iudith,

      Thanks for your thoughtful comments… Mulling over the issue this morning I came up with an idea that this blog structure may be sufficient to support.

      I will create a new post, the “Service Request Community Concierge”…. readers like you who are interested in helping the global community of Oracle developers can post comments with issues that they have discovered including reproduceable test cases. Perhaps even a separate github repository for submitting the test case scripts would be best. With each issue I can create an article detailing the issue, and I can submit the issue to MOS for evaluation. The issue post could then be updated as we hear back from Oracle Support. I know we are all busy with our lives, but together we may be able to make the time work without excessive commitment from any individual.

      Your thoughts on this idea?

      As Always, Best Regards,
      Anthony

      Like

Leave a reply to Anthony Harper Cancel reply